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FOREWORD 


This  document  Is  a  description  of  the  research  effort  of  the  fifth  year 
(Fiscal  Year  1987)  of  the  Army's  current,  large-scale  manpower  and  personnel 
effort  for  Improving  the  selection,  classification,  and  utilization  of  Army 
enlisted  personnel.  The  thrust  for  the  project  came  from  the  practical,  pro¬ 
fessional,  and  legal  need  to  validate  the  Armed  Services  Vocational  Aptitude 
Battery  (ASVAB--the  current  U.S.  military  selection/classification  test  bat¬ 
tery)  and  other  selection  variables  as  predictors  of  training  and  performance. 

The  portion  of  the  effort  described  herein  is  devoted  to  the  development 
and  validation  of  Anqy  Selection  and  Classification  measures,  referred  to  as 
"Project  A."  Project  A  Is  being  conducted  under  contract  to  the  Selection  and 
Classification  Technical  Area  (SCTA)  of  the  Manpower  and  Personnel  Research 
laboratory  (KPRL)  at  the  U.S.  Array  Research  Institute  for  the  Behavioral  and 
Social  Sciences.  This  research  supports  the  MPRl  and  SCTA  mission  to  Improve 
the  Array's  capability  to  select  and  classify  Its  applicants  for  enlistment  or 
reenlistment  by  ensuring  that  fair  and  valid  measures  are  developed  for  evalu¬ 
ating  applicant  potential  based  on  expected  Job  performance  and  usefulness  to 
the  Arn\y. 

Project  A  was  authorized  through  a  letter,  Deputy  Chief  of  Staff  for 
Operations,  "Army  Research  Project  to  Validate  the  Predictive  Value  of  the 
Armed  Services  Vocational  Aptitude  Battery,"  effective  19  November  1980;  and  a 
Memorandum,  Assistant  Secretary  or‘  Defense,  Manpower  Reserve  Affairs  &  Logis¬ 
tics  (MRA4L),  “Enlistment  Standards,"  effective  11  September  1980. 

In  order  to  ensure  that  Project  A  research  achieves  its  full  scientific 
potential  and  will  be  useful  to  the  Amy,  a  governance  advisory  group  comprised 
of  Array  general  officers.  Interservice  scientists,  and  experts  In  personnel 
measurement,  selection,  and  classification  was  established.  Members  of  the 
expert  component  provide  guidance  on  technical  aspects  of  the  research,  while 
general  officer  and  interservice  components  oversee  the  entire  research  effort, 
provide  military  judgment,  provide  periodic  reviews  of  the  project's  progress, 
results,  and  plans,  and  coordinate  within  their  commands.  Members  of  General 
Officers'  Advisory  Group  during  the  period  covered  by  this  report  included  riG 
W.  G.  O'leksy  ( DMPM )  (Chair),  MG  J.  B.  Allen,  Jr.  (DCSOPS),  MG  T.J.P.  Jones 
(FORSCOM,  DCSPER),  MG  G.  Mallory  (TRADOC,  DCS-T),  and  BG  P.  M.  Mallory  (USAREUR, 
ADCSOPS).  This  group  was  briefed  In  May  1987  on  the  results  of  the  concurrent 
validation,  the  preliminary  results  of  the  second-tour  job  analysis,  and  the 
plans  for  the  longitudinal  validation  data  collection.  Members  of  Project  A’s 
Scientific  Advisory  Group  guide  the  technical  quality  of  the  research.  During 
the  period  covered  by  this  report,  they  Included  Drs.  Philip  Bobko,  Thomas  Cook, 
Milton  Hakel  (Chair),  Lloyd  Humphreys,  Lawrence  Johnson,  Robert  Linn,  Mary 
Tenopyr,  and  Jay  Uhlaner.  This  group  was  briefed  In  March  1987  on  the  status 
of  the  second-tour  Job  analysis,  the  final  resolution  of  utility  measurement 
Issues,  and  the  reanalysis  of  the  aptitude  area  composites.  They  were  briefed 
In  September  198?  on  the  results  of  the  utility  and  construct  weighting  research 
and  the  plans  for  second- tour  criterion  measurement. 
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A  comprehensive  set  of  new  selection/classification  tests  and  job  per¬ 
formance/training  criteria  has  been  developed  and  field  tested,  and  the 
revised  tests  have  been  administered  in  a  large-scale  concurrent  validation 
data  collection  effort.  The  present  report  on  FY87  work  includes  a  reanalysis 
of  ASVAB  aptitude  area  composites  using  Project  A  criterion  measures,  a  complete 
account  of  the  work  done  to  estimate  performance  component  weights  for  the 
Batch  A  and  Batch  Z  military  occupational  specialties  (MOS)  tested  during  the 
concurrent  validation,  an  estimate  of  MOS-by-performance-level  utility  values, 
description  of  the  job  analysis  procedures  developed  for  second-tour  job 
incumbents  in  the  Batch  A  MOS,  and  the  procedures  being  used  to  administer  the 
Experimental  Predictor  Battery  to  the  longitudinal  validation  sample.  Results 
will  be  used  to  link  enlistment  standards  to  required  job  performance  standards 
and  to  more  accurately  assign  soldiers  to  Army  jobs. 

EDGAR  M.  JOHNSON 
Technical  Director, 
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PREFACE 


At  the  end  of  the  1987  fiscal  year  the  U.S.  Army  project  for  Improving  the 
Selection,  Classification,  and  Utilization  of  Army  Enlisted  Personnel  (Project 
A)  was  still  on  schedule  and  had  continued  to  meet  all  Its  major  milestones. 

The  level  of  commitment  to  a  totally  successful  project  remains  extremely  high. 
This  state  of  affairs  Is  especially  gratifying  given  that  FY87  was  the  fifth 
year  of  an  Intense  effort  that  has  continually  placed  very  high  demands  on  the 
staffs  of  the  contractor  consortia  and  the  U.S.  Hmy  Research  Institute  for  the 
Behavioral  and  Social  Sciences  (ARI).  The  prlncljal  scientist  again  wishes  to 
thank  all  the  people  who  work  on  Project  A  for  high  marks  on  both  the  "can  do" 
and  "will  do"  components  of  performance.  We  wish  also  to  express  our  continued 
appreciation  to  the  Ar^y  and  to  ARI  for  their  continued  support  and  for  their 
collegial  associations,  which  we  hope  have  been  mutually  beneficial. 


Fiscal  year  1987  was  something  of  a  transition  year  between  the  completion 
and  basic  analysis  of  the  results  of  the  1985  Concurrent  Validation  (CV)  and 
the  Longitudinal  Validation  (LV)  followup.  The  basic  CV  validity  analyses, 
performance  utility  scaling,  and  performance  components  (criterion  composite) 
weighting  efforts  were  completed.  The  administration  of  the  Experimental 
Battery  vo  the  new  1986/1987  accessions  In  the  LV  sample  was  also  virtually 
completes,  but  some  Individuals  still  had  to  complete  Advanced  Individual 
Training,  at  which  time  the  training  performance  measures  were  adnlnlstered. 
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criterion  measures.  This  part  of  the  effort  Included  the  design  of  Job 
performance  measures  for  the  noncommissioned  officers  In  their  second  tour  who 
were  part  of  the  Project  A  1983/1984  cohort. 


The  Annual  Report  for  Fiscal  Year  1987  summarizes  the  work  done  on  the 
major  components  of  this  transition  phase.  In  Chapter  2  the  Project  A  analysis 
group  reports  on  their  efforts  to  use  the  Concurrent  Validation  sample  results 
to  design  optimal  ASVAB  Aptitude  Area  composites.  Chapter  3  describes  the 
method  and  results  of  our  effort  to  capture  the  MOS-specIflc  Importance  weights 
for  the  Individual  components  of  total  performance.  Chapter  4  reports  the 
results  of  the  Project  A  procedure  for  generating  utility  values  for  different 
levels  of  performance  within  each  entry-level  MOS.  The  second-tour  NCOs  will 
be  Included  In  the  LY  followup  sample  and  Chapter  5  reports  the  method  and 
results  of  the  Job  analyses  of  second- tour  positions.  In  preparation  for  the 
development  of  NCO  criterion  performance  measures.  Finally,  the  procedure  for 
administering  the  Experimental  Battery  of  selection/classification  tests  to  new 
accessions  as  part  of  the  Longitudinal  Validation  Is  outlined  In  Chapter  6. 

This  annual  report  was  generated  by  asking  the  Individual  project  members 
responsible  for  a  particular  effort  to  sianmarlze  their  work  during  1987.  These 
drafts  were  then  edited  to  a  common  format  and  combined  with  earlier  material 
If  that  was  necessary  to  complete  the  story.  For  example,  the  work  on  composite 
weighting  and  utility  scaling  began  before  FY87  and  the  entire  account  Is 
sunmarized  here.  We  felt  It  was  better.  If  possible,  to  give  an  Inclusive 
description  of  a  major  project  activity  rather  than  report  only  what  actually 
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transpired  during  1987,  Authorship  of  the  draft  version  for  each  chapter  Is 
given  In  a  footnote  on  the  chapter  title  page.  The  editor  wishes  to  thank  the 
contrlbutoprs  for  their  valuable  materials  and  to  apologize  for  any  Injustices 
that  may  have  been  done  during  the  final  editing. 

In  sumk  FY87  laid  the  foundation  for  the  vitally  Important  longitudinal 
followup  and  the  future  analytic  work  that  will  be  necessary  to  build  an  optimal 
system  for  selectlon/classlflcatlon  decision  making.  The  future  of  Project  A 
promises  to  be  even  more  Intense  than  Its  past. 


John  P.  Campbell 
Editor 
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IMPROVING  THE  SELECTION,  CLASSIFICATION,  AND  UTILIZATION  OF  ARMY  ENLISTED 
PERSONNEL:  ANNUAL  REPORT,  198?  FISCAL  YEAR 


EXECUTIVE  SUMMARY 


Requirement; 

Project  A  Is  a  comprehensive  U.S.  Army  program  to  develop  an  Improved 
system  to  select  and  classify  enlisted  personnel.  The  system  encompasses 
675,000  persons  and  several  hundred  military  occupational  specialties  (MOS). 

The  objectives  are  to  (a)  validate  existing  selection  measures  against  both 
existing  and  project- developed  criteria  and  develop  new  measures,  and  (b) 
validate  early  criteria  (e.g.,  performance  In  training)  as  predictors  of  later 
criteria  (e.g.,  job  performance)  to  Improve  assignment  and  promotion  decisions. 


Procedure: 

With  the  Deputy  Chief  of  Staff  for  Personnel  (DCSPER)  as  sponsor,  work  on 
the  long-term  project  was  begun  In  1982.  In  the  first  stage,  relationships 
between  the  scores  applicants  made  on  the  Armed  Services  Vocational  Aptitude 
Battery  (ASVAB)  and  their  later  performance  In  training  and  first-tour  skill 
tests  were  explored  using  tile  data  for  FY81/82  Army  accessions. 

The  second  stage  was  executed  with  FY83/84  accessions  in  19  MOS,  selected 
as  representative  of  the  Army's  250+  entry-level  MOS  and  accounting  for  45 
percent  of  Army  accessions.  A  preliminary  battery  of  predictor  measures 
(perceptual,  spatial,  temperament,  interest,  and  biodata)  was  tested  with 
several  thousand  soldiers  as  they  entered  four  MOS,  revised  versions  were 
field  tested  with  nine  MOS.  The  resulting  predictor  battery  and  a  comprehensive 
set  of  job  knowledge  tests,  hands-on  job  tests,  and  performance  ratings  were 
administered  In  1985  to  9,500  soldiers  In  19  MOS  <n  the  “Concurrent  Validation." 
The  results  were  used  to  analyze  the  components  of  first- tour  performance  on 
the  job  (General  Sodiering  Skills,  MOS-Speciflc  Skills,  Leadersnip/Effort, 
Personal  Discipline.  Military  Bearing/Physical  Fitness),  and  to  compare  the 
validities  of  the  current  ASVAB  composites  and  the  added  predictor  measures  for 
predicting  Job  per fo finance. 

In  the  third  stage,  known  as  the  "Longitudinal  Validation,"  the  revised 
predictor  measures  were  used  to  test  more  than  49,000  recruits  at  the  time  they 
entered  21  MOS  In  FY86/87.  Soldiers  from  this  sample  are  being  tested  on  their 
performance  during  training  and  during  their  first  tour  on  the  job.  Soldiers 
from  the  83/84  and  86/87  sample  will  also  be  tested  on  their  second- tour 
performance. 
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Findings: 

The  effectiveness  of  current  ASVAB  Aptitude  Area  coaposltes  for  predicting 
successful  Army  performance  was  analyzed  on  the  basis  of  Concurrent  Validation 
test  data,  and  Modifications  In  composites  and  In  the  a$s1g>jnent  of  MOS  to 
Aptitude  Areas  were  proposed. 

Methods  were  developed  and  are  available  for  weighting  the  various  compo¬ 
nents  of  the  criterion  tests  to  provide  composite  scores  of  job  performance 
that  could  be  used  for  classification/selection  purposes. 

Methods  were  developed  and  are  available  for  estimating  the  utility  of 
various  combinations  of  MOS  and  performance  level  as  a  selection  too!.  Utility 
values  at  five  performance  levels  were  calculated  for  273  entry-level  MOS. 

The  predictor  and  criterion  tests  developed  during  the  earlier  stages  are 
now  being  used  to  measure  the  performance  of  a  large  sample  of  soldiers  during 
their  first  tour. 


Utilization  of  Findings: 

The  Project  A  tests  for  predicting  and  measuring  training  and  job  per¬ 
formance  are  being  used  in  both  current  and  long-range  research  programs  that 
are  expected  to  make  the  Army  more  effective  in  matching  the  requirements  for 
first-tour  enlisted  manpower  with  the  personnel  resources  that  are  available 
to  the  Army . 
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IMPROVING  THE  SELECTION,  CLASSIFICATION,  AND  UTILIZATION 
OF  ARMY  ENLISTED  PERSONNEL:  ANNUAL  REPORT,  1987  FISCAL  YEAR 

OVERVIEW  OF  PROJECT  A 

Project  A  Is  a  comprehensive  long-range  research  and  development  program 
the  U.S.  Army  has  undertaken  to  develop  an  Improved  system  for  selecting  and 
classifying  enlisted  personnel.  The  Army’s  goal  Is  to  Increase  Its  effectiveness 
In  matching  first-tour  enlisted  manpower  requirements  with  available  personnel 
resources  through  use  of  new  and  Improved  selection/classification  tests  that 
will  validly  predict  carefully  developed  measures  of  Job  performance.  The  proj¬ 
ect  addresses  the  Army's  675,000-person  enlisted  personnel  system  encompassing 
several  hundred  military  occupations. 

The  program  began  In  1980,  when  the  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences  (AR1)  started  planning  the  extensive  research 
needed  to  develop  the  desired  system.  In  1982  ARI  selected  a  consortia,  led 
by  Human  Resources  Research  Organization  (HtanRRO)  and  Including  American  In¬ 
stitutes  for  Research  (AIR)  and  Personnel  Decisions  Research  Institute  (PDRI), 
to  undertake  the  9-year  project.  It  \r.  utilizing  the  services  of  40  to  50  ARI 
and  consortium  researchers  working  colleglally  In  a  variety  of  professional 
specialties.  The  Project  A  objectives  are  to 

•  Val Idate  existing  selection  measures  against  both  existing  and 
project-developed  criteria  (Including  both  Army-wide  job  performance 
measures  based  on  rating  scales  and  direct  hands-on  measures  of  MOS- 
speclflc  task  performance). 

•  Develop  and  validate  new  selection  and  classification  measures  of 
other  human  attributes  that  underlie  success  on  the  Job. 

•  Determine  the  relative  utility  to  the  Army  of  different  performance 
levels  across  MOS. 

The  research  design  Incorporates  three  main  stages  of  data  collection  and 
analysis  in  an  Iterative  progression  of  development,  testing,  evaluation,  and 
further  development  of  selectlon/classlficatlon  Instruments  (predictors)  and 
measures  of  job  performance  (criteria).  In  the  first  iteration,  file  data  from 
fiscal  years  (FY)  1981/1982  were  evaluated  to  explore  relationships  between 
scores  of  applicants  on  the  Armed  Services  Vocational  Aptitude  Battery  (ASYAB) 
and  their  later  performance  In  training  and  their  scores  on  first-tour  Skill 
Qualification  Tests  (SQT). 

For  the  ensuing  research,  19  Military  Occupational  Specialties  (MOS)  were 
selected  as  a  representative  sample  of  the  Army's  250+  entry-level  MOS.  The 
selection  was  based  on  an  Initial  clustering  of  MOS  derived  from  rated  similar¬ 
ities  of  job  content.  These  MOS  account  for  about  45  percent  of  Army  accessions 
and  provide  sample  sizes  large  enough  so  that  race  and  sex  fairness  can  be 
empirically  evaluated  In  most  MOS. 
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In  the  second  Iteration,  a  Concurrent  Validation  design  was  executed  with 
FY83/84  accessions.  A  "Preliminary  Battery"  of  perceptual,  spatial,  temperament, 
Interest,  and  biodata  predictor  measures  was  developed  and  tested  with  several 
thousand  soldiers  as  they  entered  four  MOS.  The  data  from  this  sample  were 
then  used  to  refine  the  measures,  with  further  exploration  of  content  and 
format.  The  revised  set  of  measures  was  field  tested  to  assess  reliabilities, 
"fakabll Ity,"  practice  effects,  and  other  factors.  The  resulting  predictor 
battery,  the  "Trial  Battery,"  was  administered  together  with  a  comprehensive 
set  of  job  performance  Indexes  based  on  job  knowledge  tests,  hands-on  job 
samples,  and  performance  rating  measures,  In  the  Concurrent  Validation  during 
the  sunnier  and  fall  of  1985.  The  results  of  the  Concurrent  Validation  were 
used  to  form  five  "constructs"  of  the  components  of  performance  and  to  report 
to  the  Army  the  validity  of  the  ASVAB  for  predicting  job  performance  as  well  as 
the  incremental  validities  of  the  Trial  Battery  components  over  ASYAfi  predictors. 

On  the  basis  of  this  experience,  the  "Trial  Battery"  was  revised  as  the 
"Experimental  Predictor  Battery,"  which  in  turn  Is  being  administered  in  tne 
third  Iteration,  the  Longitudinal  Validation  stage,  which  began  In  the  late 
summer  of  1986.  All  measures  are  being  administered  In  a  true  predictive 
validity  design.  About  50,000  soldiers  across  21  MOS  are  Included  In  the  FY86- 
87  Initial  administration  and  the  subsequent  measurement  of  first-tour  perfor¬ 
mance.  About  3,500  of  these  soldiers  are  expected  to  be  available  for  second- 
tour  performance  measurement  In  FY91.  Three  MOS  (19K,  29E,  and  96B)  were 
added  to  the  original  19,  and  one  of  the  original  MOS  (76W)  was  dropped. 

For  administrative  purposes,  Project  A  Is  divided  Into  five  research  tasks: 
Task  1,  Validity  Analyses  and  Data  Base  Management,  Task  2.  Developing  Predictors 
of  Job  Performance;  Task  3,  Developing  Measures  of  School/Training  Success; 

Task  4,  Developing  Measures  of  Army-Wide  Performance;  Task  5,  Developing  M0S- 
Speclflc  Performance  Measures. 

Activities  during  the  first  four  years  of  Project  A  were  reported  as  follows 
FY83,  AR1  Research  Report  1347  and  its  Technical  Appendix,  ARI  Research  Note  83- 
37,  FY84,  ARI  Research  Report  1393  and  two  related  reports,  ARI  Technical  Report 
66C  and  ARI  Research  Note  85-14,  FY85,  ARI  Technical  Report  746  and  ARI  Research 
Note  97-54;  FY86,  ARI  Technical  Report  792  and  ARI  Research  Note  88-36.  Tne 
present  FY87  report  is  supplemented  by  ARI  Research  Note  (in  preparation). 

These  reports  list  other  publications  on  specific  Project  A  activities. 
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IMPROVING  THE  SELECTION,  CLASSIFICATION,  AND  UTILIZATION 
OF  ARMY  ENLISTED  PERSONNEL: 

ANNUAL  REPORT*  1987  FISCAL  YEAR 


Chapter  1 

HIGHLIGHTS  OF  FISCAL  YEAR  1987  AND  OBJECTIVES  OF  THIS  REPORT 


This  report  Is  Intended  to  be  a  suaaary  of  the  major  activities  In  the 
research  program  of  the  Army  Selection  and  Classification  Project  (Project 
A)  during  fiscal  year  1987  (FY87).  Project  A  Annual  Report?  for  the  four 
preceding  years  concentrated  respectively  on  a  description  of  research 
planning  and  basic  preparation  (FY83).  the  initial  stages  of  the  development 
of  new  predictor  and  criterion  tests  (FY84),  a  comprehensive  summary  of  the 
process  and  products  of  predictor/criterion  development  and  field  testing 
(FY85),  and  the  basic  analysis  and  results  of  the  Concurrent  Validation 
tests  (FY86). 

Briefly  restated,  the  operational  objectives  of  Project  A  are  to 

•  Develop  new  measures  of  job  performance  that  the  Army  can  use  as 
criteria  against  which  to  validate  selection/classification 
measures. 

e  Validate  existing  selection  measures  against  both  existing  and 
project-developed  criteria. 

o  Develop  and  validate  new  selection  and  classification  measures 
of  other  human  attributes  that  underlie  success  on  the  job. 

•  Develop  a  utility  scale  for  different  performance  levels  across 
military  occupational  specialties  (MOS). 


In  addition,  a  number  of  related  and  derivative  research  objectives  have 
been  addressed  in  the  overall  research  program. 


STATE  OF  PROJECT  A  AS  OF  30  SEPTEMBER  1987 

In  FY87  the  Army  Selection  and  Classification  Project  entered  a 
critical  period.  As  shown  in  Figure  1.1,  which  is  a  sunaary  of  the  original 
research  design  schedule,  the  project  should  be  (a)  in  the  midst  of 
finishing  the  administration  of  the  Experimental  Predictor  Battery  to  more 
than  50,000  new  accessions  in  21  MOS  (the  86/87  cohort);  (b)  planning  for 
the  longitudinal  followup  and  performance  assessment  of  soldiers  from  this 
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Figure  1.1.  The  overall  research  design  for  Project  A 


sample  In  1968/89;  (c)  planning  for  the  second-tour  followup  of  the  Concur* 
rent  Validation  (CV )  sample  (i.e.,  the  83/84  cohort);  and  (d)  continuing  the 
analysis  of  the  CV  sample  data.  At  the  close  of  FY87,  five  years  after  It 
began.  Project  A  was  still  on  its  original  schedule. 

By  virtue  of  being  on  schedule,  the  following  major  objectives  of  the 
project  have  been  accomplished: 

•  A  4-hour  prediction  battery  of  new  experimental  selection/cl  os¬ 
sification  tests  has  been  developed  and  validated  on  a  concurrent 
sample  of  9,450  incumbents  drawn  from  19  MOS. 

•  To  provide  criteria  for  validation  purposes,  a  12-hour  performance 
assessment  battery  has  been  developed,  field  tested,  and  adminis¬ 
tered  to  the  CV  sample. 

•  Results  from  the  CV  administration  have  been  used  to  formulate  a 
five-factor  model  of  first-tour  performance  for  enlisted  person¬ 
nel.  The  differential  prediction  across  criterion  factors  and 
across  jobs  has  been  analyzed  and  parameter  estimates  are  avail¬ 
able  for  use  in  simulation  runs  of  the  enlisted  personnel  alloca¬ 
tion  system  (EPAS). 

•  Finally,  for  purposes  of  developing  a  complete  selection/classifi¬ 
cation  prediction  system,  criterion  component  weights  and  MOS-by- 
performance- level  utility  values  have  been  estimated,  using  expert 
judgnient  scaling  techniques. 


Detailed  information  concerning  the  development  of  the  predictor 
battery  and  the  first-tour  job  and  training  performance  measures  is  given  in 
the  Annual  Report  for  FY85  (Campbell,  1985).  Results  from  the  Concurrent 
Validation  are  presented  in  the  Annual  Report  for  FY88  (Campbell,  1986a). 
Work  accomplished  during  FY87  is  described  in  this  report,  as  outlined 
below. 


wnicjii  ur  mid  nt.ru ki 

The  work  suoraarized  in  this  report  was  performed  in  several  different 
parts  of  the  Project  A  organization.  Consequently,  the  chapters  cover  a 
series  of  more  discrete  topics  than  was  the  case  in  the  reports  for  FY85  and 
FY86. 


The  significant  organizational  events  that  occurred  during  FY87  are 
summarized  in  this  chapter  (Chapter  1).  A  reanalysis  of  the  current 
Aptitude  Area  composites  based  on  the  Armed  Services  Vocational  Aptitude 
8attery  (ASVAB),  using  Project  A  criterion  measures,  will  be  described  and 
the  results  reported,  in  Chapter  2. 
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Chapter  3  Is  a  complete  account  of  the  work  done  to  estimate  weights 
for  the  various  components  of  performance  for  the  MOS  tested  In  the 
Concurrent  Validation  phase.  Both  the  weights  and  a  procedure  for  using 
them  to  estimate  composite  criterion  scores  are  now  available. 

A  similar  report  for  estimating  HOS-by-pcrformance- level  utility  values 
Is  n  In  Chapter  4.  A  two-step  procedure  was  used  to  estimate  a  ratio 
sew  «.  utility  value  for  five  levels  of  performance  In  each  of  273  Army 
entry-level  MOS. 

Chapter  5  describes  the  job  analysis  procedures  and  Initial  results  for 
second-tour  job  Incumbents  In  the  nine  MOS  (Batch  A)  used  In  Project  A's 
Initial  test  development  and  field  tests  with  soldiers  of  the  83/84  sample. 
These  job  analyses  are  the  basic  preparation  for  the  development  of  NCO  job 
performance  measures  that  will  be  carried  out  In  FY88. 

finally,  Chapter  6  outlines  the  procedures  being  used  to  administer  the 
Experimental  Predictor  Battery  to  the  Longitudinal  Validation  sample  of  21 
MOS.  As  of  the  end  of  FY87,  the  predictor  battery  had  been  administered  to 
approximately  48,000  new  86/87  accessions  being  processed  at  reception 
centers. 

In  sun,  Chapters  2,  3,  and  4  address  Issues  that  are  relevant  for  the 
operational  use  of  a  modified  selectlon/classlflcatlon  system  by  the  Army. 
Chapters  5  and  6  pertain  to  the  second  major  data  collection,  the  Longitudi¬ 
nal  Validation  and  second-tour  follow-up;  Chapter  7  provides  a  brief 
description  of  what  remains  to  be  done. 


ORGANIZATIONAL  EVENTS  IN  FYB7 

The  composition  of  the  Project  A  governance  groups  and  the  organization 
of  the  research  and  oversight,  staff  at  the  end  of  FY87  are  shown  In  Figures 
1.2  and  1.3. 

Among  the  significant  organizational  events  for  Project  A  during  FY87 
were  the  following: 

•  Or.  Marvin  H.  6oer  retired  from  HuoRRO  in  February  1987.  His 
position  as  Project  Director  was  filled  by  Mr.  James  H.  Harris. 

•  Two  of  the  original  Project  A  tasks  —  Developing  Predictors  of 
Job  Performance  (Task  2)  and  Developing  Measures  of  School/Train¬ 
ing  Success  (Task  3)  —  ended  as  of  30  September  1987.  The 
respective  staffs  for  each  task  prepared  final  reports  of  Task 
activities. 

•  Administration  of  the  Experimental  Predictor  Battery  to  a  sample 
of  approximately  50,000  recruits  in  21  MOS  was  virtually  com¬ 
pleted.  Results  from  these  tests  provide  the  basic  data  for  use 
in  the  Longitudinal  Validation  process,  through  measurement  of 
first-  and  second-tour  performance  of  soldiers  from  this  sample. 
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Figure  1.2. 


Project  A  organization  as  of  3Q  September  1987. 
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Flgiir*  1.3.  Project  A  mnegemmt  group  is  of  30  Sepfeobor  1987. 


e  Work  was  completed  on  two  aspect!  of  the  process  of  selecting  and 
classifying  personnel  —  a  system  for  weighting  various  criterion 
components,  and  a  set  of  estimated  values  for  HOS-by-perforwance- 
level  utility. 

•  The  Scientific  Advisorv  Group  and  the  General  Officers*  Advisory 
Group  each  Independently  endorsed  the  second-tour  criterion 
measurement  plan.  That  plan  calls  for  the  development  of  second- 
tour  measures  for  al 1  Batch  A  NOS. 

e  Work  mis  completed  on  an  extensive  analysis  of  data  generated  from 
Project  A  and  Skill  Qualification  Tests  (SQT)  to  estimate 
potential  benefits  from  possible  realignment  of  Aptitude  Area 
composites  or  reassignment  of  HOS  to  different  coe^osit*:;. 
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Chapter  2 

PROJECT  A  ANALYSIS  OF 
ASYAS  AFTITUOE  AREA  COMPOSITES  1 

During  FY87  the  Project  A  staff  completed  an  extensive  analysis  of  file 
data  and  project-generated  data  for  the  purpose  of  recommending  possible 
changes  In  the  ASVAB  Aptitude  Area  composites.  Current  Army  enlistment 
policies  require  an  applicant  to  pass  two  separate  cognitive  abilities 
screens  based  on  ASVAB  subtests.  There  are  10  subtests: 

-  Word  Knowledge  (MC) 

-  Paragraph  Comprehension  (PC) 

-  Arithmetic  Reasoning  (AR) 

-  Mathematical  Knowledge  (MO 

-  Mechanical  Comprehension  (HC) 

-  Coding  Speed  (CS) 

-  Numerical  Operations  (NO) 

-  General  Science  Information  (6$) 

-  Auto/Shop  Information  (AS) 

-  Electronics  Information  (El) 

In  practice,  the  first  two  subtests  are  combined  into  a  single  Verbal  (VE) 
score,  leading  to  a  total  of  nine  subtests. 

One  composite  of  these  subtests,  known  as  the  Armed  Forces  Qualifica¬ 
tion  Test  (AFQT),  Is  used  by  all  Services  as  a  general  measure  of  train- 
ability.  To  qualify  for  enlistment  Into  the  Army,  an  applicant  must  achieve 
at  least  a  minimum-level  AFQT  score.  To  qualify  for  enlistment  bonuses,  an 
applicant  must  score  in  the  upper  50th  percentile  range  (based  on  norms  for 
tiie  1980  youth  population). 

In  addition  to  the  AFQT,  the  Army  uses  nine  other  composites  of  ASVAB 
subtests,  called  Aptitude  Area  (AA)  composites,  to  assess  applicant  qualifi¬ 
cation  for  enlistment  into  particular  MOS.  Nearly  all  enlisted  MOS  are 
associated  with  one  of  these  nine  composites,  and  an  applicant  must  achieve 
a  specified  minimum  score  on  the  associated  composite  to  qualify  for  enlist¬ 
ment  Into  the  MOS.  These  nine  AA  composites  are  unigue  to  the  Army.  The 
other  Services  use  a  similar,  but  separate,  system  of  composites  to  qualify 
applicants  for  particular  specialties.  The  nine  Army  composites  are: 

-  Clerical  (CL) 

-  Combat  (CO) 


^his  chapter  Is  based  on  an  Initial  draft  by  Lauress  L.  Wise  of  the 
American  Institutes  for  Research.  Material  was  also  drawn  from  Wise, 
McHenry,  Rossmelssl,  and  Oppler  (1986). 
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-  Electronic*  (EL) 

-  Field  Artillery  (FA) 

-  General  Maintenance  (GM) 

•  Mechanical  Maintenance  (W4) 

-  Operators/Food  (OF) 

-  Survelllance/Coaaunlcatlon  (SC) 

»  Skilled  Technical  (ST) 

As  one  of  the  first  research  efforts  under  Project  A,  In  ?Y84,  the 
staff  conducted  analyses  of  ASVAB  validities,  using  a  combination  of  train¬ 
ing  measures  and  scores  from  the  Skill  Qualification  Tests.  Two  of  the  AA 
composites  (Clerical  and  Survelllance/Communlcetlon)  were  modified  by  the 
Army  as  a  result  of  these  analyses  (McLaughlin,  Rossmeissl,  Vise,  Brandt,  A 
Wang,  1584). 

In  FY87,  Project  A  staff  conducted  new  analyses  of  the  validities  of 
the  current  AA  composites  and  various  alternatives  for  predicting  perfor¬ 
mance  In  enlisted  MOS.  The  three  phases  to  these  analyses  were 

•  Consideration  of  redefining  the  current  AA  composites, 
based  primarily  on  analyses  of  19  Project  A  MOS  and  using 
Project  A  criterion  measures. 

•  Evaluation  of  options  for  significantly  reducing  the 
number  of  composites,  based  on  both  Project  A  and  SQT 
criteria. 

•  Identification  of  optimal  reass igrunents  of  MOS  to  current 
or  slightly  modified  AA  composites,  based  primarily  on  SQT 
criteria. 

These  three  phases  were  chronological  as  well  as  logical  steps  in  the 
analyses.  The  procedures  and  results  of  each  phase  will  be  discussed  in 
turn.  Results  will  then  be  summarized  and  conclusions  stated. 


CONSIDERATION  OF  REDEFINING  THE  CURRENT  AA  COMPOSITES 

This  part  of  the  analysis  began  with  the  use  of  Project  A  data  as 
criteria  for  evaluating  alternative  definitions  for  each  of  the  current  AA 
composites.  The  results  were  presented  to  the  Military  Testing  Association 
In  November  1986  (Wise  et  al.,  1586). 

AA  composites  are  defined  as  unweighted  sums  of  four  or  fewer  of  the 
standardized  subtest  scores.  There  are  255  such  possible  composites  (126 
using  four  subtests,  84  using  three,  36  using  two,  and  9  using  a  single 
subtest).  All  of  the*  were  evaluated  using  data  from  the  Project  A 
Concurrent  Validation. 
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The  CV  data  included  the  new  Project  A  job  performance  measures  applied 
to  more  than  9.000  soldiers  in  19  different  MOS.  The  MOS  used  In  the  CV 
phase  were: 


11B  Infantryman 

126  Combat  Engineer 

136  Cannon  Crewsan 

16S  MANPADS  Crewman 

19E  Armor  Crewman 

27E  TOW/Dracon  Repairer 

31C  Single  Channel  Radio  Operator 

516  Carpentry/Masonry  Specialist 

54E  Chemical  Operations  Specialist 

55B  Amain it Ion  Specialist 


036  Light  Wheel  Vehicle  Mechanic 
64C  Motor  Transport  Operator 
67N  Utility  Helicopter  Repairer 
71L  Administrative  Specialist 
76W  Petroleum  Supply  Specialist 
76Y  Unit  Supply  Specialist 
91A  Medical  Specialist 
946  Food  Service  Specialist 
956  Military  Police 


Project  A  performance  measures  have  been  organized  Into  five 
"constructs, “  reflecting  dimensions  of  soldier  performance  (Campbell, 
1986a).  Four  of  these  constructs  (General  Soldiering  Proficiency,  Effort 
and  Leadership,  Personal  Discipline,  and  Physical  Fitness  and  Military 
Bearing)  are  the  same  for  all  MOS.  The  fifth  construct.  Core  Technical 
Proficiency  (CTP),  covers  aspects  of  job  performance  unique  to  each  MOS  and 
is  appropriate  for  validating  AA  scores,  which  are  used  as  job-specific 
selection  criteria. 


Table  2.1  shows  CV  sample  sizes  by  MOS.  race,  and  gender  and  presents 
the  mean  scores  and  standard  deviations  for  the  ASVAB  subtests  and  the  CTP 
criterion. 


Criteria  Used  to  Evaluate  Composites 


Four  separate  criteria  were  used  in  evaluating  current  and  alternative 
composites:  (a)  predictive  validity,  (b)  fairness  to  blacks  and  females, 
(c)  classification  efficiency,  and  (d)  face  validity.  Each  criterion  is 
described  briefly  before  the  results  are  discussed. 


Predictive  Validity.  The  correlation  of  each  composite  with  the  CTP 
score  was  adjusted  for  restriction  of  range  due  to  explicit  selection.  A 
multivariate  correction  fro*  l  a*? ’ey  (Lord  S  novicfc,  1368,  P-  146)  wi s  used 
with  each  of  the  ASVAB  subtests  treated  as  a  separate  selection  variable. 
The  result  was  used  as  the  measure  of  predictive  validity. 


No  adjustment  was  made  for  "shrinkage"  In  cross-validation  since 
separate  regression  coefficients  were  not  estimated.  For  evaluation  of  the 
current  composites,  this  is  entirely  appropriate.  Because  we  did  pick  among 
a  large  number  of  alternative  composites  on  the  basis  of  the  data  at  hand, 
some  shrinkage  should  be  expected  for  the  alternatives  that  appear  most 
extreme.  Conventional  shrinkage  formulas  do  not  handle  this  situation,  so 
our  best  approach  Is  to  be  somewhat  conservative  in  adopting  new 
alternatives  to  the  existing  composites. 
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Fairness  tc  Blaeks  and  fggju.  Separate  regression  equations  were 
computed  by  race  and  gender  where  there  were  at  least  50  examinees  In  each 
subgroup.  (For  exaaple,  for  NOS  11B,  90  blacks  were  aaong  those  tested  but 
data  were  complete  for  fewer  than  50  so  the  regression  equation  for  race  was 
not  computed.)  Both  slope  and  Intercept  differences  were  Identified.  A 
single  overall  Measure  of  the  difference  between  the  separate  equations  was 
defined  as  the  difference  In  predicted  values  for  an  AA  score  of  100  (the 
estimated  aean  for  the  1980  norm  population).  Since  selection  cutoffs 
ranged  between  85  and  110  for  the  H0S  In  question,  a  score  of  100  was 
selected  as  being  In  the  heart  of  the  critical  region  for  evaluating  the 
selection  fairness  of  alternative  coaposites.  Differences  In  the  prediction 
equations  at  points  significantly  below  or  above  this  value  would  have 
little  lapact  on  deteralning  applicant  qualification.  The  difference  In 
predicted  values  was  converted  to  a  £  score  by  dividing  by  the  standard 
error  of  the  estlaate  of  the  difference  (Pothoff,  1964). 

Classification  Efficiency.  The  Brogden  Index  (defined  as  the  square 
root  of  the  average  validity  times  the  square  root  of  one  ainus  the  average 
of  the  Intercorrelitlons  aaong  the  coaposites)  was  used  as  a  aeasure  of 
classification  efficiency  (Brogden,  1946).  -This  statistic  Is  an  indicator 
of  the  predictions  of  differences  in  an  Individual's  expected  performance 
across  Jobs. 

Face  Validity.  Face  validity  is  not  easily  quantifiable,  but  Is  More 
appropriately  used  as  e  check  of  the  "reasonableness”  of  the  results.  It  Is 
our  attempt  to  check  purely  eapirical  results  against  soae  conception  of 
theory.  Me  would  be  uncomfortable,  for  exaaple,  with  results  Indicating 
that  Auto/Shop  Information  (AS)  is  an  isportant  predictor  for  clerical  Jobs, 
but  quite  comfortable  with  AS  as  an  important  predictor  for  vehicle 
Mechanics. 

Results  Froa  Evaluation  of  Coaposites 

Table  2.2  shows  validities,  Brogden  Indexes  of  classification 
efficiency,  and,  where  appropriate,  race  and  gender  £  statistics  for  each 
contending  AA  composite.  Separate  statistics  are  shown  for  each  applicable 
H0$,  and  unweighted  averages  of  the  vanities  and  £  statistics  are  shown 
for  the  H0S  cluster  as  a  whole.  Each  row  corresponds  to  a  different 

TL-  /1..A  .  4  -  X _ AL.  - A. - -  .14.  .  .i4L  J.  A. - 
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alternative  coaposites  are  labeled  Ai  through  A9.  Data  are  also  shown 
(labeled  PR)  for  the  Cl  and  SC  coaposites  replaced  in  1984  after  our  earlier 
analyses  (McLaughlin,  et.  al.,  1984).  In  soae  cases  where  another  current 
coaposite  has  a  higher  average  validity  than  the  operational  composite  for 
the  cluster,  data  are  shown  in  rows  labeled  according  to  the  other  coaposite 
(this  occurs  In  the  CO  and  OF  coaposites).  The  results  presented  in  Tabic 
2.2  are  discussed  separately  for  each  of  the  current  AA  coaposites. 

Clerical  (CL).  The  current  CL  coaposite  has  an  average  validity  that 
Is  as  high  or  higher  than  any  other  alternative.  It  does,  however, 
underpredict  feaale  performance  In  the  two  clerical  specialties  (NOS  71L  and 
76Y)  where  separate  predictions  were  generated.  The  addition  of  either  the 
NO  or  the  CS  subtest  significantly  reduces  the  underprediction  for  females 
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Tab  It  t.t.  Validity.  Cultural  Fairnass.  and  Classification  Efficiency 
Indicators  for  Currant  and  Other  ASVA8  Composite* 
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without  significantly  reducing  validity.  Adding  NO  (Numerical  Operations) 
reduces  underprediction  the  most,  while  adding  CS  (Coding  Speed)  has  the 
greatest  face  validity  and  results  In  slightly  greater  classification 
efficiency.  A  slightly  different  pattern  was  found  for  NOS  76W;  the 
addition  of  AS  increases  validity  for  predicting  76W  performance ,  while 
decreasing  validity  for  predicting  NOS  71L  and  76Y  performances.  Notwith¬ 
standing  these  differences,  the  current  and  primary  alternative  CL 
composites  predict  performance  in  all  three  clerical  NOS  quite  well. 

Combat  (CO).  The  current  CO  has  high  validity  for  each  of  the  three 
NOS  examined. Some  gain  in  validity  would  be  realized  by  substituting  GS 
(General  Science  Information)  for  CS  and,  perhaps,  also  exchanging  NK 
(Mathematical  Knowledge)  for  AR  (Arithmetic  Reasoning).  The  inclusion  of  GS 
would  improve  prediction  in  all  three  NOS.  The  greater  contribution  of  GS 
also  is  i  tional  in  light  of  the  increasing  technical  sophistication  in  the 
systems  us  d  in  combat  specialties.  Adding  GS  would  also  reduce  the  small 
degree  of  overprediction  of  the  performance  of  blacks. 

Electronics  (EL).  The  current  EL  composite  does  quite  well  for  the  one 
EL  specialty  examined.  Substitution  of  NO  for  one  or  both  of  the 
quantitative  subtests  would  increase  both  predictive  validity  and 
classification  efficiency,  but  not  to  any  practical  extent. 

field  Artillery  (FA).  Neither  the  current  FA  nor  any  alternative 
composite  appears  to  have  a  very  high  validity  for  predicting  NOS  138 
performance.  The  fact  that  several  other  current  composites  have  slightly 
higher  validities  for  predicting  138  performance  than  does  the  current  FA 
composite  encourages  consideration  of  alternative  composites.  Substitution 
of  NO  and  AS  (Auto/Shop  Information)  for  CS  and  m  would  yield  the  most 
significant  gains.  Such  substitution  also  significantly  reduces 
overprediction  for  blacks. 

General  Maintenance  (GNl.  Very  high  validities  were  found  for  the 
current  GM  composite  for  Both  HOS  51B  and  55B.  Very  slight  gains  might 
result  from  substituting  VE  (Verbal)  for  El  (Electronics  Information)  or 
from  simply  dropping  El,  but  these  gains  would  be  offset  by  small  increases 
in  over  prediction  of  blacks'  performance  and  slightly  lower  classification 
efficiency  estimates. 

Nechanical  Maintenance  (MH).  High  validities  were  found  for  the 
current  Mi  composite  in  predicting  both  NOS  63B  and  67N  performance.  Small 
gains  in  the  prediction  of  63B  performance  and  increased  classification 
efficiency  would  result  from  dropping  the  NO  subtest. 

Operators/Food  (OF).  The  OF  results  closely  parallel  the  CL  results. 
Female  performance  is  significantly  underpredicted  for  NOS  94B.  Another 
specialty,  NOS  64C,  shows  a  somewhat  different  pattern  of  validities,  with 
AS  again  (and  not  surprisingly)  adding  significantly  to  the  predictive 
validity  of  this  one  specialty.  In  fact,  the  same  composites  appear  optimal 
for  both  the  CL  and  the  OF  NOS  —  AR*VE-*ttC*NO  for  NOS  I6S  and  94B  (as  for 
NOS  71L  and  76Y)  and  AR+YE+MC+AS  for  NOS  64C  (as  for  75W).  Substituting  AR 
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*nd  NK  for  AS  And  NC  (Nechanical  Comprehension)  Mould  significantly  reduce 
underprediction  of  female  perfomsnce  for  NOS  94B  while  increasing  overall 
validity. 

Survtlllance/Co— unication  (SCI.  A  high  predictive  validity  was  found 
for  the  current  SC  composite.  Some gain  In  validity,  along  with  a  slight 
Increase  In  classlf Icatior  efficiency,  would  result  If  MC  were  replaced  by 
NO.  This  would  lead  to  a  small  increase  In  the  underprediction  of 
performance  for  blacks.  If  MC  were  also  substituted  for  AR,  the  saw  gains 
In  validity  and  classification  efficiency  could  be  obtained  along  with  a 
decrease  In  underprediction  of  blacks'  performance. 

Skilled  Technical  (ST).  The  current  ST  Is  a  true  Amy  composite  —  It 
js  all  that  it  can  be. It  has  a  higher  average  validity  than  any  possible 
alternative,  and  It  shows  no  significant  differences  In  the  prediction  of 
performance  for  blacks  and  females. 

Conclusions 

The  main  findings  from  these  analyses  were  as  follows: 

e  The  current  AA  composites  had  generally  high  validities  for 
predicting  technical  proficiency  In  enlisted  NOS. 

e  Some  changes  were  identified  that  might  lead  to  small  Increases  in 
validity  or  small  reductions  in  over-  or  underprediction  for 
particular  subgroups  of  applicants. 

e  For  two  of  the  AA  composites,  Cl  and  OF,  different  predictor 
composites  appeared  optimal  for  predicting  performance  in 
different  NOS  currently  assigned  to  the  same  composite. 

The  last  finding  suggested  the  need  to  consider  reassigning  NOS  to 
different  current  AA  composites,  and  perhaps  changing  the  number  of 
different  composites.  Since  analyses  of  possible  reassignments  would  have 
to  include  all  entry-level  NOS,  not  just  the  Project  A  NOS,  additional  data 
were  needed  to  continue  exploration  of  possible  modifications  of  composites. 

Earlier  research  had  demonstrated  a  reasonable  correlation  between 
results  from  Project  A  criterion  measures  and  SQT  scores  (Arabian  &  Nason, 
1986).  Therefore,  the  Project  A  staff  analyzed  SQT  scores  for  1983  and  1984 
as  a  cross-check  and  extension  of  the  validities  obtained  with  Project  A 
criteria.  Table  2.3  shows  the  adjusted  (for  range  restriction)  validities 
for  predicting  SQT  scores  for  each  of  the  NOS  for  which  SQT  data  were 
available.  Validities  were  estimated  for  each  of  the  current  AA  scores, 
including  the  GT  composite  that  is  not  currently  used  for  HOS-specIflc 
selection,  and  for  a  specific  set  of  alternatives  to  each  of  the  current  AA 
scores.  Composites  used  by  the  other  Services  are  also  included,  to  see 
whether  any  greater  consistency  across  Services  could  be  achieved. 
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Tables  2.4  and  2.5  show  the  degree  to  which  subgroups  (females  and 
blacks,  respectively)  are  underpredicted  by  a  common  regression  line. 

Entries  In  these  tables  are  the  difference  between  the  subgroup  and  common 
regression  lines  at  an  AA  score  of  100.  These  differences  are  In  SQT  score 
units— a  percentage-correct  Metric  running  from  zero  to  100.  Analyses  are 
reported  only  for  MOS  for  which  SQT  data  were  available  for  at  least  50 
■ewers  of  each  subgroup.  Analyses  by  gender  were  necessarily  emitted  for 
NOS  that  were  closed  to  women.  Only  differences  that  are  significant  at  the 
.05  level  are  shown.  Very  few  of  the  differences  shown  are  of  any  practical 
significance  (i.e.,  sore  than  a  few  percentage  points). 

The  following  conclusions  were  drawn  fro*  the  above  results: 

•  Alternative  strategies  for  reducing  the  nuaber  of  different 
coaposltes  should  be  evaluated. 

e  SQT  data  should  be  analyzed  further  to  evaluate  potential  gains 
froa  reassigning  NOS  to  different  AA  coaposltes. 


EVALUATI03  OF  0PTIQRS  FOR  A  REDUCED  RUBER  OF  AA  COMPOSITES 

Identification  of  options  for  reducing  the  nuaber  of  AA  coaposltes 
began  with  an  examination  of  the  dimensions  of  predicted  performance  scores 
for  the  19  Project  A  MOS  In  the  Concurrent  Validation  stage.  For  each  MOS, 
optimal  predictor  weights  were  Identified  for  predicting  job  performance 
froa  the  nine  ASVA8  subtests.  Predicted  performance  scales  were  calculated 
both  for  the  Core  Technical  Proficiency  (CTP)  factor  based  on  Project  A 
data,  and  for  the  SQT.  For  each  of  these  two  criteria,  the  matrix  of 
Intercorrelations  among  the  predicted  scores  for  the  different  MOS  was 
factor  analyzed. 

The  predicted  scores  for  the  different  NOS  were  all  highly  correlated, 
leading  to  a  large  first  factor  in  the  matrix  of  correlations  among  these 
scores.  There  was  some  evidence  for  a  second  factor,  but  no  support  for  any 
further  factors.  Figures  2.1  and  2.2  show  a  plot  of  the  loadings  of  the 

?  rediction  equations  for  each  MOS  on  the  first  two  factors  after  rotation, 
he  vertical  factor  consists  almost  exclusively  of  the  MC,  AS,  and  El 
composites  and  has  been  labeled  “technical."  The  second  factor  consists  of 
both  verbal  and  quantitative  subtests  and  has  been  labled  "academic." 

Both  figures  show  a  continuum  running  from  NOS  638  at  one  end  to  MOS 
71L  at  the  other.  Table  2.6  shows  the  optimal  regression  coefficients  for 
predicting  the  CTP  score  In  each  of  these  MOS  when  the  nine  ASVAB  subtests 
are  clustered  into  four  relatively  distinct  composites.  (Reducing  the 
colinearity  among  the  predictors  in  this  way  leads  to  greater  stability  in 
regression  coefficient  estimates.)  These  are  the  same  four  composite  scores 
used  as  predictors  In  the  basic  Concurrent  Validation  analysis  (cf. 

Campbell,  1987).  As  the  table  shows,  the  Nearest  difference  between  jobs 
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Figure  2.1.  Mot  of  loading*  on  fectors  underlying  correlations  eaong 
predicted  scores:  Project  A  Core  Technical  Proficiency 
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Table  2.6 

Regression  Coefficients  for  Predicting  Coro  Technical  Proficiency  (CTP) 
Free  ASVA6  Construct  Scores 


Composite/ 

MOS  « 
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Quanti¬ 
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6S,VE 


Speed  JlteitMU 
CS,«0  T  Q  V  S 


CL:  Clerical 
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.421 
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.208 

.152 

0 
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1 

1 

76W 

339 

.479 

.265 

.283 

.206 

.032 
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1 
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76Y 
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.453 

.070 

.374 
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.137 
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2 
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1 

CO: 

Combat 

0 

UB 
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.464 

.285 
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.216 

.128 

2 

1 

1 

12B 
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.449 

.422 
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.149 

-.032 
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1 

1 

0 

19E 

394 

.331 

.393 

.199 

.078 

-.047 

2 

1 

0 

0 

EL: 

Electronics 

• 

27E 

123 

.505 

.245 

.161 

.362 

.132 

2 

1 

2 

1 

FA: 

Field  Artillery 

.082 

13B 

464 

.132 

.248 

.060 

.053 

3 

1 

1 

1 

CM: 

6encral 

Maintenance 

SIB 

69 

.672 

.268 

.198 

*18 

.192 

2 

1 

2 

1 

S5B 

203 

.572 

.332 

.085 

.4i$ 

-.004 

2 

0 

2 

0 

MM: 

Mechanical  Maintenance 

63B 

478 

.466 

.700 

-.010 

-.032 

.045 

4 

0 

0 

0 

67N 

238 

.658  • 

.641 

.117 

.0S4 

.039 

4 

0 

0 

0 

OF: 

Operators/Food 

16S 

338 

.262 

.061 

.391 

.208 

-.001 

0 
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1 

0 

64C 

507 

.313 
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-.056 

-.020 
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1 

0 

0 

94B 
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.170 

.201 

0 

2 
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1 

ac: 

»  *  % 
i'jrvei  i 

1  ance/Conaaun  1  ca  i 

iunS 

'  31C 

289 
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.202 

1 
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1 

1 

ST: 

Skilled 

Technical 

54E 

340 

.489 

.247 

.326 

.265 

-.087 

1 

1 

1 

0 

91A 
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.544 

.207 
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.349 

.200 

1 

G 

2 

1 

$58 

597 

.399 

.147 

.233 

.226 

.126 

1 

1 

1 

1 

ASAB  Subtests 
AR  -  Arithmetic  Reasoning 
AS  -  Auto/Shop  Information 
CS  -  Coding  Speed 
El  -  Electronics  Information 


6S  -  6eneral  Science  Information 
MC  -  Mechanical  Couprehension 
MK  -  Mathematical  Knowledge 
HO  -  numerical  Operations 
VE  -  Verbal 


36 


Is  In  the  degree  of  Inclusion  of  the  'technical*  subtests  of  the  ASVAB 
(e.g.,  NC,  AS,  end  El).  For  NOS  638  end  67N  (which  currently  use  the  m 
composite),  only  technical  subtests  receive  significant  weight.  For 
clerlcal/adalnlstratlve  Jobs  (e.g.,  NOS  71L  ana  76Y),  the  technical  subtests 
receive  virtually  no  weight. 

When  SQT  scores  rather  than  Project  A  measures  of  technical 
proficiency  are  used  as  the  criterion  for  prediction  equations,  the  results 
are  substantially  the  same.  NOS  638  and  71L  still  anchor  the  extreses.  The 
loadings  for  NOS  118  are  aidway  between  these  two  extreaes,  with  other 
coabat  NOS  (138,  16S,  and  19E)  loading  soaewhat  sore  with  the  technical  than 
with  the  adalnlstrative  NOS.  (No  SQT  data  were  available  for  NOS  91A/B.) 
There  were  a  few  notable  differences,  but  these  primarily  concerned  NOS 
(e.g.,  16S  or  27E)  with  relatively  saell  sample  sizes  for  one  or  the  other - 
of  the  types  of  criteria. 

Even  though  there  appeared  to  be  only  two  relevant  predictor  dimen¬ 
sions,  the  array  of  Jobs  was  fairly  continuous  in  tern  of  their  relative 
eaphasis  on  these  two  diaenslons.  Balancing  all  considerations,  six* 
clusters  of  Jobs  were  chosen  as  the  prlaary  alternative  to  the  current  nine. 
Figure  2.3  shows  the  current  and  proposed  predictor  coaposltes. 

In  addition  to  the  aechanlcal  and  adalnlstrative  clusters  at  the 
extreaes,  analysts  Identified  a  'general*  cluster  of  Jobs  for  which  approxi¬ 
mately  equal  weight  on  the  two  types  of  predictors  was  optlaal,  and  a 
‘technical  support*  cluster  of  jobs  for  which  soaewhat  wore  weight  for  the 
general  cognitive  predictors  In  comparison  to  the  technical  predictors  was 
optlaal. 

Two  other  clusters  of  Jobs  in  this  aiddle  range  were  also  identified 
and  kept  separate.  First,  Jobs  eaphasizlng  electronics  were  separated 
because  there  was  only  weak  evidence  that  the  El  subtest  had  any  greater 
validity  alone.  Second,  coabat  Jobs  were  kept  In  a  separate  cluster.  Here 
too,  there  was  not  strong  evidence  for  different  patterns  of  validity  aaong 
the  current  ASVAB  subtests.  However,  the  Introduction  of  measures  being 
developed  In  Project  A  (Including  Coabat  Interest  In  particular)  should 
provide  a  basis  for  differential  prediction  for  this  laportant  group  of 
Jobs. 

Next,  the  Project  A  NOS  were  grouped  into  the  six  clusters  of  Jobs  and 
an  optlaal  predictor  coaposite  was  determined  for  each  cluster.  Table  2.7 
shows  validity  and  subgroup  (race  and  gender)  fairness  analyses  for  the 
current  (old)  and  alternative  (new)  coaposltes,  using  Project  A  CV  data  as 
criteria. 

One  conclusion  froa  the  analyses  of  the  Project  A  data  was,  again,  that 
the  evidence  for  generalizing  froa  the  current  set  of  NOS  clusters  was  not 
at  all  convincing.  Therefore,  SQT  data  were  again  used  as  a  basis  for  anal¬ 
yzing  an  expanded  set  of  NOS.  For  these  analyses,  the  1985  SQT  scores  were 
merged  with  the  scores  from  1983  and  1984  used  in  the  earlier  SQr  analyses, 
permitting  analyses  of  some  additional  NOS  and  Increasing  the  accuracy  of 
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NCTE:  2  ■  double  value  for  this  subtest; 

.5  •  only  half  value  for  this  subtest.. 


Figure  2.3.  Definition  of  current  and  proposed  Aptitude  Area  cosposltes 
In  tents  of  ASVAE  standardized  subtests. 
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results  for  other  MQS.  Appendix  8  shows  the  validities  and  subgroup  differ¬ 
ences  for  the  current  and  alternative  composites  for  each  MOS.  (Entries  In 
this  appendix  are  grouped  within  CMh  by  the  proposed  new  opposite.) 

In  addition  to  differences  in  validity  and  fairness,  classification 
efficiency  was  also  considered.  Table  2.8  shows  the  Intercorrelations  for 
the  current  AA  composites  and  alternative  AA  composites,  and  also  shows  the 
Brogdan  Index  values  of  classification  efficiency.  The  new  composites  did 
have  noticeably  higher  classification  efficiency  values.  This  Increase  was 
due  In  part  to  small  gains  in  average  validity  (.61  to  .63)  and  In  part  to  a 
reduction  In  the  average  correlation  among  composites  (from  .82  to  .89). 

The  reduction  In  the  average  correlation  among  composites  was  not  due  simply 
to  a  reduction  in  the  number  of  composites.  For  the  currant  composites,  the 
minimum  correlation  (between  the  Cl  and  MH  composites)  is  .80;  for  the 
alternative  composites,  the  minimum  correlation  (between  NM  and  KA)  Is  .65. 

In  sum,  the  results  of  the  second  phase  of  AA  validity  analyses 
supported  the  following  conclusions: 

•  There  was  little  evidence  for  any  significant  benefit  from 

Increased  validity  or  fairness  associated  with  the  new  composites. 

e  Changing  to  an  entirely  new  set  of  Aptitude  Area  composites 
would  entail  major  Army  actions  that  do  not  appear  to  be 
justified  or.  the  basis  of  the  modest  estimates  of  benefit. 

e  Most  of  the  Increase  in  classification  efficiency  recorded  for 
the  new  composites  (e.g.,  Increased  validity  and  decreased  corre¬ 
lation  among  composites)  could  be  realized  by  changing  the  sub¬ 
tests  used  in  the  existing  Mechanical  Maintenance  (ffi;  composite. 

§  Most  of  the  rest  of  the  benefit  from  the  new  composites  could  be 
realized  by  reassigning  some  MOS  to  a  different  ore  of  the  current 
composites  without  reducing  the  number  of  different  composites 
used. 


IDEMTIFICATl&ti  OF  OPTIMAL  REASSZGNMEMTS  OF  MOS  TO  AA  COMPOSITES 

In  accordance  with  recommendations  from  the  Project  A  Scientific 
Advisory  Group,  the  project  staff  reanalyzed  the  SQT  data  to  identify  MOS 
for  which  reassignment  to  a  different  composite  from  among  the  current 
composites  (or  the  proposed  new  MM  composite)  appeared  Indicated.  Table  2.9 
summarizes  these  results  by  MOS  within  career  management  field  (CMF).  A 
complete  listing  of  the  separate  sample  estimates  for  each  MOS  Is  shown  as 
Appendix  C. 

A  list  of  specific  changes  was  prepared  and  briefed,  along  with  the 
proposed  change  In  FM  (Figure  2.4),  to  the  Project  A  General  Officers* 
Advisory  6roup  on  1  May  1987.  A  few  further  changes  were  proposed  to  cover 
reserve  component  MOS  and  MOS  for  which  no  data  were  available  (on  the  basis 
of  similarity  to  other  MOS).  The  final  results  of  these  analyses  were 
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Table  2.8 


Classification  Efficiency:  Correlations  for  Currant  and  Proposed  Ccn^osltes, 
Estimated  for  1980  Youth  Population 
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TabU  2,9 

Naan  Validities  for  AA  Composites  by  NOS  Within  Caroar  N&nagaaant  Fields 
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Table  2.9  (Continued) 

Mean  Validities  for  AA  Coa^osltas  by  NOS  Within  Career  Nanegemnt  Field* 
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Table  2.1  (Continued) 

Naan  Validities  for  AA  Cooposites  by  NOS  Within  Career  Managaaent  Fields 
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Table  2.9  (Continued) 

Naan  Validities  for  AA  Composites  by  NOS  Within  Caroar  Management  Flalds 
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Mechanical  Maintenance  Aptitude  Area  Composite 

Current  (Ifl):  AS  (Auto/Shop  Information) 

PC 

JL, 

(Mechanical  Comprehension) 

El 

+ 

(Electronics  Information) 

NO 

(Numerical  Operations) 

Proposed  (NM): 

2 

AS  (Auto/Shop  Information) 

▼ 

MC 

▲ 

(Mechanical  Comprehension) 

▼ 

El 

(Electronics  Information) 

Figure  2.4.  Proposed  change  In  the  Mechanical  Maintenance  Coaqroslta. 


briefed  once  wore  at  the  Project  A  Analysis  Group  In-Progress  Review  (IPR) 
meeting  in  July  1937.  Table  2.10  shows  the  specific  changes  that  were 
proposed.  Subsequent  to  this  Meeting,  briefings  on  the  proposed  changes 
were  prepared  for  each  of  the  proponent  schools.  ARI  staff  have  presented 
these  briefings  where  they  were  requested. 


SUNKARY  AND  CONCLUSIONS 

The  Most  significant  finding  from  this  research  was  that  the 
existing  AA  composites  do  a  reasonably  good  job  of  predicting  subsequent  job 
performance.  In  the  end,  only  relatively  minor  changes  to  the  existing  set 
of  composites  were  proposed.  These  included  a  change  in  the  formula  for 
computing  the  Mechanical  Maintenance  composite  and  the  reassignment  of 
several  MOS  to  different  composites  for  classification  purposes. 

A  second  finding  was  that  the  relevant  dimensionality  of  the  current 
predictors  Is  small.  Only  two  relevant  dimensions  were  found.  This  leaves 
a  great  deal  of  room  for  additional  predictors  (e.g.f  spatial,  psychomotor, 
or  Interest  measures)  that  might  be  of  significant  benefit  in  Increasing 
classification  efficiency. 


46 


Tablt  2.10 


Proposed  Changes  In  Assignwnt  of  NOS  to  Aptitude  Area  Cooposltas 
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Tab la  2.10  (Continued) 

Proposed  Changes  In  Assignment  of  NOS  to  Aptitude  Area  Composites 
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A  third  finding  was  the  Importance  of  SQT  data  for  monitoring  the 
appropriateness  of  different  predictors  for  different  MOS.  Given  the 
changing  nature  of  many  NOS,  It  would  seem  prudent  to  plan  for  periodic 
reanalyses  as  new  SQT  data  become  available. 

Finally,  development  of  estimates  of  the  potential  benefit  of  AA 
changes  that  go  beyond  indexes  of  validity  or  classification  efficiency 
would  be  desirable.  6iven  the  constraints  on  assigning  applicants  to  NOS 
and  the  uncertain  influence  of  applicant  choice,  an  accurate  overall  index 
seems  unlikely.  It  might  be  preferable  to  develop  indirect  indicators  of 
the  adequacy  of  the  AA  composites.  The  number  of  recruits  who  do  not  have 
enough  ab'lity  to  complete  training  or  the  number  who  do  not  achieve 
adequate  job  proficiency  might  be  of  particular  interest.  The  Project  A 
longitudinal  data  should  provide  a  much  stronger  basis  for  describing  the 
impact  of  alternative  selection  composites. 
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Chapter  S 

WEIGHTING  CRITERION  COMPONENTS  FOR  A  COMPOSITE  INDEX1 

\ 


The  data  fro*  the  Concurrent  Validation  staple  have  been  used  to  revise 
and  develop  *ore  completely  a  aodel  of  job  performance  for  entry-level 
performance  in  terms  of  five  basic  components  (General  Soldiering  Skills, 
MOS-Specific  Technical  Skills,  Leadership/Effort,  Personal  Discipline, 
Military  Bearing/Physical  Fitness).  This  process  mas  described  In  the 
Project  A  Annual  Report  for  FY  1986  (Campbell,  1886a;  also  Carfbell,  1986b, 
and  Wise,  Campbell,  McHenry,  A  Hanser,  1986). 

Results  have  shown  that  each  of  the  components  can  be  predicted  with 
considerable  validity  and  that  the  validity  of  the  different  predictor 
domains  varies  systematically  across  criterion  components.  Yet  to  be 
determined  is  how  a  composite  index  of  performance  can  be  formed  and  what 
the  validity  of  the  Trial  Battery  Is  for  each  Job,  when  just  one  composite 
Indicator  of  performance  is  used. 

This  chapter  describes  research  conducted  to  determine  the  best  method 
to  weight  the  importance  of  the  job  performance  components  in  an  overall 
composite  index  of  performance.  Weighting  judgments  for  each  Project  A  MOS 
were  then  gathered  from  noncommissioned  officers  (RCOs)  and  officers 
familiar  with  each  MOS.  Analyses  of  these  data  are  presented  in  the  final 
sections  of  the  chapter. 


BACKGROUND 

Several  methods  are  available  for  assigning  weights  to  dimensions  of 
performance  in  such  a  way  that  they  reflect  the  dimensions'  relative 
importance  to  overall  performance.  Four  procedures  that  have  been 
emphasized  In  the  literature  are  !a)  the  Two-Factor-at-a-Time  conjoint 
procedure;  (b)  the  Full-Profile  conjoint  procedure;  (c)  the  Kelly  Bids 
system;  and  (d)  the  Kane  method. 

In  a  conjoint  procedure  the  respondents  are  asked  to  rank  order,  rate, 
or  otherwise  choose  among  two  or  more  sets  of  profile  descriptions  that  vary 
along  the  dimensions  of  interest.  The  relative  weights  for  the  dimensions 
can  be  Inferred  from  the  relationships  between  the  dimensio..  values  built 
into  the  descriptions  and  the  rank  orders  or  ratings  (the  dependent 
variable)  of  the  profiles.  The  Two-Factor-at-a-Time  and  the  Full-Profile 
approaches  have  been  generally  used  in  conjoint  procedures. 

The  Two-Factor-at-a-Time  is  also  referred  to  as  the  Trade-off  procedure 
(Johnson,  1974).  In  this  procedure  the  performance  factors  are  evaluated  on 
a  two-at-a-time  basis.  The  evaluators  are  usually  asked  to  rank  the  various 
combinations  of  each  pair  from  most  preferred  to  least  preferred  (Green  A 


1Thf s  chr~ter  Is  based  on  Sadacca,  deVera,  DIFazio,  and  White  (1986) 
*nd  Sadacca,  Campbell,  White,  and  DIFazio  (1988). 
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Srlnlvasan,  1978).  The  advantages  of  using  this  procedure  are  that  It  Is 
simple,  reduces  Information  overload,  and  lends  Itself  to  mail  questionnaire 
administration.  It  does,  however,  have  soae  limitations.  It  has  been 
criticized  as  being  unrealistic  because  rtere  are  other  factors  that  must 
•Iso  be  considered  In  the  overall  evaluation.  Some  researchers  (6reen, 

1974;  Johnson  ft  VanOyk,  1975)  have  pointed  out  that  the  total  number  of 
required  evaluations  Is  quite  large  when  there  are  multiple  levels  within 
the  dimensions;  In  these  circumstances  the  respondents  may  attend  to  one 
dimension  first  before  considering  the  otter  (Johnson,  1974). 

The  Full-Profile  approach  attempts  to  address  some  of  the  limitations 
of  the  Two-Factor-at-a-Time  procedure,  following  the  same  procedure  but 
utilizing  the  complete  set  of  factors  In  the  descriptions.  It  gives  a  more 
realistic  description  of  the  stimuli  being  Judged  by  defining  the  levels  on 
•11  of  the  factors,  «nd  possibly  taking  Into  account  the  potential 
environmental  correlations  between  the  factors  In  real  stimuli  (Green  ft 
Srlnlvasan,  1978).  It  Is,  however,  not  devoid  of  limitations.  Information 
overload  Is  highly  likely  as  the  number  of  factors  In  the  profile  Increases. 
Furthermore,  tne  respondents  may  simplify  the  task  by  Ignoring  variations  In 
the  less  Important  factors  or  by  simplifying  the  factor  levels  themselves 
(Green  ft  Srlnlvasan,  1978).  For  these  reasons,  use  of  this  procedure  Is 
generally  limited  to  five  or  six  factors. 

The  measurement  scale  used  for  these  conjoint  procedures  Is  either  non¬ 
metric  (paired  comparisons,  rank  order)  or  metric  (rating  scales  assuming 
Interval  scales,  ratio  scales  obtained  by  constant-sum  paired  comparisons). 
For  the  Two-Factor,  the  non-metric  scale  Is  more  appropriate  because  the 
rank  order  of  the  cells  In  a  trade-off  table  need  not  depend  on  the  levels 
of  the  missing  factors,  except  If  the  attributes  are  correlated  (Green  ft 
Srlnlvasan,  1978). 

The  effectiveness  of  these  two  procedures  has  been  evaluated  by  several 
researchers.  Montgomery,  Hitt ink,  and  Glaze  (1977)  reported  that  the  Two- 
Factor  procedure  yielded  higher  predictive  validity.  Their  research  focused 
on  Job  choices  made  by  business  graduates  and  used  a  total  of  eight 
attributes.  In  a  study  of  commuters'  choice  of  transportation  modes  tint 
varied  along  nine  attributes,  Alpert,  Betak,  and  Golden  (1978)  reported 
better  goodness-of-flt  for  the  Two-Factor  procedure.  Jain,  Aclto,  Malhotra, 
and  Mahajan  (1978),  on  the  other  hand,  reported  that  the  two  methods  yielded 
approximately  the  same  level  of  cross-validity  In  the  context  of  choosing 
checking  accounts  offered  by  various  banks  when  the  accounts  were  described 
via  five  attributes.  Oppedijk  van  Veen  and  Jeazley  (1977)  found  that  the 
utilities  determined  by  the  two  methods  were  roughly  similar  In  the  context 
of  a  durable  goods  product  class  when  using  three  attributes. 

In  the  Kelly  Bids  system  for  weighting  purposes,  the  respondents  are 
asked  to  allocate  100  points  across  tne  criterion  dimensions  on  the  basis  of 
their  relative  importance.  Schmidt  (1977)  found  this  procedure  better  than 
others  because  the  focus  is  on  the  hypothetical  "true*  criterion. 
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Kane  (1960)  maintained  that  observability  and  uncertainty  should  also 
be  considered  critical  in  all  appraisal  situations.  He  therefore  proposed 
the  Kane  method  for  assigning  weights  to  performance  factors.  An  important 
aspect  to  this  procedure  Is  the  designation  of  a  level  of  specificity  for 
assigning  importance  weights  (e.g.,  task  level)  prior  to  any  activity. 

The  respondent*  are  then  asked  to  identify  the  coeponent  having  the  least 
importance  for  measuring  overall  effectiveness;  this  component  is  assigned  a 
weight  of  1.0.  The  respondents  are  then  asked  to  compare  the  regaining 
factors  to  the  least  important  coeponent,  assigning  weights  to  reflect  how 
eany  tines  nore  Inportant  each  factor  is  in  comparison  to  the  least 
important  factor. 

All  four  procedures  for  assigning  weights  to  performance  factors  have 
been  shown  to  work  well  in  a  variety  of  settings.  The  appropriateness  of 
the  methodology  depends  to  a  great  extent  on  the  purposes  and  the  type  of 
factors  and  variables  of  the  research  endeavor.  Consequently,  before 
proceeding  to  the  actual  determination  of  NOS  weights,  the  Project  A  staff 
conducted  a  series  of  exploratory  studies  with  the  various  procedures  to 
determine  which  methods  would  be  most  suitable  for  this  project. 


PILOT  TESTS  OF  METHODS  FOP  WEIGHTING  CRITERION  COMPONENTS 

Three  piiot  experiments  were  conducted  tc  select  the  procedures  to  be 
used  in  weighting  performance  constructs  (components)  for  Project  A.  The 
primary  focus  in  these  experiments  was  on  the  weighting  procedures  them¬ 
selves,  not  on  the  weights  of  the  constructs  for  a  given  NOS.  Cur  interest 
In  conducting  the  experiments  was  in  selecting  one  or  more  procedures  for 
weighting  the  components  of  performance  that  would  be  acceptable  to  the  Army 
and  would  yield  a  reliable,  valid  set  of  weights  for  each  of  the  sampled  KOS 
when  the  procedures  were  applied  by  the  appropriate  subject  matter  experts 
(SMEs).  The  three  pilot  experiments  were  related  in  the  sense  that  the 
weighting  procedure  selected  as  a  result  of  the  first  experiment  was  also 
used  in  the  second  and  third  experiments  to  further  evaluate  that  and  other 
procedures.  The  experiments  and  their  results  will  be  described  briefly 
prior  to  describing  the  actual  component  weighting  procedure. 

Exssrisar.t  1:  Procedure  end 

Sixteen  Army  officers  stationed  at  Fort  Meade,  Maryland,  and  Fort 
Monroe,  Virginia,  participated  in  the  first  experiment.  Their  task  was  to 
assign  relative  weights  to  six  performance  constructs  for  three  MOS  — 
Infantryman  (11B),  Light  Wheel  Vehicle  Mechanic  (638),  and  Adainistrative 
Specialist  (71L). 

At  the  time  the  experiment  was  conducted  in  the  surmer  of  1985,  the 
Project  A  performance  constructs  for  a  job  performance  model  had  not  yet 
been  selected.  Therefore,  a  plausible  set  of  six  constructs  whose  weights 
might  be  expected  to  vary  considerably  was  used  for  the  experiment.  The  six 
performance  constructs  were  (a)  dependability,  (b)  MOS-specific  task 
performance,  (c)  MOS  knowledge,  (d)  military  bearing,  (e)  performance  under 
adverse  conditions,  and  (f)  performance  on  common,  general  soldiering  tasks 
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(e.g.,  putting  cn  a  gas  mask).  The  construct  weights  for  the  three  HOS  were 
assigned  by  the  officers  under  a  replicated  3x3  Graeco-Latin  square  design 
(Figure  3.1)  in  which  three  weighting  procedures  were  used  under  three 
different  ■ilitary  scenarios. 

All  three  procedures  involved  direct  judgments  of  the  relative  weight 
that  each  performance  construct  should  receive  in  forming  an  overall 
composite  performance  score.  In  procedure  Af  the  officers  were  first  asked 
to  rank  order  the  six  constructs,  and  then  to  assign  IOC  points  to  the 
first-ranked  construct  and  scale  the  other  constructs  accordingly  (this  Is  a 
variant  of  the  Kane  method).  In  procedure  B,  the  officers  were  instructed 
to  divide  100  points  among  the  six  constructs  In  a  manner  that  reflected  the 
relative  weight  that  should  be  given  the  constructs.  In  procedure  C,  IS 
pairs  of  the  six  constructs  were  presented  in  a  paired-comparison  protocol; 
the  order  of  presentation  followed  the  optimization  procedure  worked  out  by 
Ross  (1934).  The  officers'  task  was  to  divide  100  points  between  the  two 
constructs  being  judged  In  any  given  pair. 

The  judgments  were  made  in  the  context  of  three  different  scenarios 
(Figure  3.2).  The  scenarios  described  respectively  a  peacetime  condition, 
a  period  of  heightened  tensions,  and  a  wartime  setting  In  which  hostilities 
had  just  broken  out.  The  site  (i.e.,  Europe)  of  the  three  scenarios  was  the 
same. 

After  completing  the  construct  weighting  judgments,  each  officer  used 
four  7-point  scales  to  evaluate  the  weighting  methods  on  the  following 
dimensions: 

(1)  Acceptability  to  the  Army. 

(2)  Ease  :f  making  the  judgments  called  for  by  the  method. 

(3)  Their  confidence  in  the  validity  of  the  Judgments  made. 

(4)  The  amount  of  agreement  with  other  workshop  participants 
that  could  be  expected. 

The  relevant  mean  ratings  across  the  four  dimensions  are  shown  in  Table  3.1. 
After  the  officers  completed  rating  the  methods,  an  informal  discussion  was 
held  to  solicit  their  opinions  about  the  methods. 

The  design  permitted  testing  for  the  significance  of  differences  in 
mean  ratings  on  the  four  dimensions  for  procedures  and  for  scenarios,  and 
for  any  Procedure  by  Scenario  interactions.  None  of  the  main  effects  due  to 
the  scaling  procedure  or  scenario  were. significant.  However,  significant 
(£<.05)  Procedure  by  Scenario  interactions  were  obtained  for  the  scales 
showing  acceptability  to  the  Army  and  the  raters'  confidence  in  their 
Judgment,  and  for  the  average  of  the  four  scales.  Procedure  A  (in  which  100 
points  were  assigned  to  the  first-ranked  construct)  had  particularly  low 
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Scaling  Methods:  Maximum  100  points  (A),  divide  100  points  (B), 

paired  comparison  (C). 

Military  Scenario:  Wartime  (a),  period  of  heightened  tensions  (b), 

peacetime  (c). 


Number  of 


SuMasli, 

BQillfi 

nos  m 

MQLX1L 

2 

Aa 

Bb 

Cc 

1 

Be 

Ca 

Ab 

1 

Cb 

Ac 

Ba 

HOS  63W 

Ml  211, 

mm 

2 

Aa 

Bb 

Cc 

2 

Be 

Ca 

Ab 

2 

Cb 

Ac 

Ba 

MU1L 

mm 

2 

Aa 

Bb 

Cc 

2 

Be 

Ca 

Ab 

2 

Cb 

AC 

Ba 

Figure  3.1. 


Replicated  Graeco-Latin  Square  design. 
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PEACETIME  SCENARIO 


Europe  is  In  the  peacetime  condition  currently  prevailing  there. 
Your  Corps'  mission  is  to  defend  and  maintain  the  host  country's 
border  should  Mar  break  out.  The  potential  enemy  approximates  a 
combined  arms  Army  and  has  nuclear  and  chemical  capability.  Air 
parity  does  exist.  The  Corps  has  personnel  and  equipment  suffi¬ 
cient  to  make  it  mission  capable  for  training  and  evaluation.  The 
training  cycle  includes  periodic  field  exercises,  command  and 
maintenance  inspections,  ARTEP  evaluations,  and  individual  soldier 
training/SQT  testing. 


HEI6KTEMED  TENSIONS  SCENARIO 

Europe  is  In  a  period  of  heightened  tensions.  There  is  an 
Increasing  probability  that  hostilities  will  break  out  in  the  next 
several  months.  Your  Corps'  mission  is  to  defend  ar.d  maintain  the 
host  country's  border  should  war  break  out.  The  potential  enemy 
approximates  a  combined  eras  Army  and  has  nuclear  and  chemical 
capability.  Air  parity  does  exist.  The  Corps'  training  and  other 
preparatory  activities  have  been  substantially  Increased.  Most 
combat  and  associated  support  units  are  participating  in  frequent 
field  exercises.  Most  units  are  being  actively  resupplied. 


WARTIME  SCENARIO 

Hostilities  have  broken  out  in  Europe  and  your  Corps'  combat  units 
are  engaged.  Your  Corps'  mission  is  to  defend,  then  reestablish, 
the  host  country's  border.  Pockets  of  enemy  airborne/heliborne 

ana  guerilla  elements  are  Operating  throughout  the  CcrpS  SSCtor 

area.  Limited  initial  and  reactive  chemical  strikes  have  been 
employed  but  nuclear  strikes  have  not  been  initiated.  Air  parity 
does  exist. 


Figure  32.  Three  different  military  scenarios. 


Table  3.1 


Experiment  1:  Keen*  Ratings*3  of  Kino  Weighting  Procedure/ScenaHo 
Coot 1 nations 


Scenario 


Emsdms. 

EffteStlHB 

Heightened  Tensions 

Herl  ine 

A. 

Maximum  -  100  points 

2.85 

4.75 

4.79 

B. 

Divide  100  points 

4.95 

5.12 

4.20 

C. 

Paired  Comparison 

4.62 

4.60 

4.35 

*  Separate  Beans  based  on  ratings  of  five  or  six  officers, 
b  Seven-point  rating  scales  In  which  1  •  Low  and  7  »  High. 


ratings  when  combined  with  the  peacetime  scenario,  but  had  relatively  high 
ratings  when  combined  with  the  wartime  and  heightened  tensions  scenarios. 

The  officers  generally  expressed  preference  for  procedures  A  and  C  over 
procedure  B,  and  thought  that  the  time  they  spent  in  Procedure  B  in  Baking 
sure  that  the  sum  of  their  weights  equaled  100  detracted  from  their  ability 
to  judge  the  relative  importance  of  the  performance  factors.  It  was  also 
evident  that  if  a  larger  number  of  constructs  were  ultimately  identified, 
procedures  B  and  C  could  become  fairly  onerous. 


The  officers  also  expressed  a  general  preference  for  the  heightened 
tensions  and  wartime  scenarios  over  the  peacetime  scenario  as  the  setting 
for  the  judgments.  From  the  discussion,  it  also  seemed  that  a  heightened 
tensions  scenario  would  evoke  a  more  uniform  frame  of  reference  across  the 
many  different  kinds  of  subject  matter  experts  providing  the  MOS  construct 
weights  than  a  wartime  scenario  would,  unless  the  wartime  scenario  was  made 
cuite  specific.  However,  specificity  in  the  scenario  Couiu  produce  unwanted 
dependency  of  the  construct  weiahts  on  particular  elements  in  the  scenario, 
which  could  detract  from  the  validity  of  the  weighted  composite  as  an 
overall,  general  measure  of  MOS  performance. 


Experiment  2;  Procedurt  and  Results 

The  second  pilot  experiment  was  conducted  in  the  winter  of  1985  at  Fort 
Bragg..  North  Carolina,  using  two  4-hour  workshops.  One  workshop  was 
attended  by  15  officers,  the  other  by  15  NCOs.  The  workshop  participants 
were  asked  to  weight  five  performance  constructs  for  the  Infantry  HOS:  (a) 
demonstrating  commitment  to  the  Army,  (b)  technical  proficiency  and 
knowledge,  (c)  physical  fitness  and  military  bearing,  (d)  performance  under 
adverse  conditions,  and  (e)  maintaining  and  servicing  weapons  and  equipment. 


Each  participant  used  the  three  different  weighting  Methods  described 
In  the  following  Instructions: 

(1)  Rank  order  the  five  constructs,  assign  100  points  to  the 
first-ranked  construct,  and  then  scale  the  other  constructs 
accordingly  (same  as  procedure  A  In  Experiment  1). 

(2)  Based  upon  their  scores  on  the  separate  constructs,  rank 
order  25  Infantrymen  In  order  of  their  overall  performance. 

For  each  Infantryman,  a  different  set  of  performance  scores 
on  the  five  constructs  was  given  on  7-point  scales  that 
range  from  the  lowest  level  of  performance  to  the  highest. 

A  sample  profile  Is  shown  In  Figure  3.2. 

(2)  Based  upon  their  scores  on  two  constructs,  rank  order  10 

sets  of  13  Infantrymen  In  order  of  their  overall  performance. 

In  each  set,  the  performance  scores  on  two  constructs  a re 
given  on  the  same  7-polnt  scales  used  In  the  second  method 
above.  A  set  of  13  Infantrymen  Is  given  for  each  of  the  10 
possible  pairs  of  the  five  constructs.  (See  Figure  3.4.) 

The  second  and  third  methods  are  variants  of  the  conjoint  approach  to 
scaling  In  which,  Instead  of  obtaining  the  relative  Importance  of  the 
performance  constructs  directly,  the  Judges'  weights  for  the  performance 
constructs  are  inferred  from  the  rank  order  they  give  sets  of  hypothetical 
soldiers  whose  performance  on  the  constructs  has  been  systematically  varied. 
Multiple  regression  weights  ere  calculated  fro*  the  Interrelationships 
between  the  rank  orders  provided  by  the  Judges  and  the  performance  construct 
levels  given  in  the  performance  descriptions.  In  the  paired-comparison 
method,  these  regression  weights  are  then  used  to  derive  the  construct 
weights,  using  a  ratio  scaling  procedure  described  by  Torgersot;  (1958,  pp. 
105-112).  This  procedure  results  In  a  set  of  scale  values  or  weights  for 
the  constructs  whose  geometric  mean  is  equal  to  1.0. 

The  Judgments  were  made  in  the  context  of  a  worldwide  increase  in 
tensions  (Figure  3.5).  The  weighting  methods  were  applied  in  counter¬ 
balanced  order  by  the  15  participants  in  each  workshop.  After  completing 
each  method,  the  participants  rated  the  method  on  the  four  7-point  scales 
ustJ  in  the  first  experiment. 

Table  3.2  presents  the  mean  rating*,  given  the  three  weighting  methods 
by  the  30  workshop  participants,  along  with  the  results  of  analysis  of 
variance  tests  of  the  significance  of  the  method  mean  differences.  The 
ratings  clearly  favored  the  direct  estimation  method,  while  the  full-profile 
conjoint  method,  which  involved  rank  ordering  the  descriptions  of  25 
hypothetical  Infantrymen,  generally  received  the  lowest  ratings.  A  breakout 
of  these  ratings  by  type  of  judge  indicated  that  both  the  officers  and  the 
NCOs  generally  preferred  the  direct  estimation  method  most  and  the  full- 
profile  conjoint  method  least. 
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Soldier 


tank  ftre*r 
Overall  k#f» 


"OS:  Jn'an:ry«*<i  (111) 


A.  DEMONS THAT l«;  COW ITMC AT  TO  TX  AWT 

Maintaining  Amy  traditions,  spirit  and  fellowship. 


Snows  Iks  of  dddlcstton 
t»  Aroy  traditions  and 
values. 

Generally  supports  Amy 
traditions  and  valves. 

Snows  constant  devotion 
to  Army  tradition  end 

values. 

1  2 

1  ©  » 

A  1 

1.  PHYSICAL  riTXSS  and  niutaat  appeaxace 

Maintaining  Military  stsnoards  of  pfiysclal  fitness; 

Mlntalning  proper  military  appearance  and 

Standards  Of  cleanliness  and  groaning. 

Maintains  self  in  poor 
physical  condition,  falls 
to  neet  Military  standards 
for  drass  and  personal 
hygiene. 

Heats  Aroy  standards  of 
physical  fitness.  Dresses 
neatly  ane  mats  Amy 
standards  of  personal 
hygiene. 

Exceeds  Amy  standards 
and  eipectations  set  tor 
physical  fitness. 

Maintains  excellent 
personal  hygiene  end 
proper  appearance. 

1  2 

0  4  * 

A  7 

C.  MAINTAINING  AND  SKV1CHC  WEAPOAS  AND  EQUIPMENT 
Keeping  weapons  and  equipment  clean  and  serviced 
and  prepared  Tor  the  field. 


falls  to  perform  or  jPerfjms  routine  cnects 

’•properly  per'oms  checks  land  preventive  maintenance 
end  preventive  memtenencejon  weapons  and  ocuipmen;. 

on  weapons  on;  edu'pment.  1 

Always  teen  es$i;*ei 
weapons  anj  eouirx't  in 
read/*for.inspe:t:tn 
COno’tlP". 

1  2  J  ©  i  A  7 

D.  TECHNICAL  PROFICIENCY  AND  WOW  LEDGE 

E'fectiveness  in  applying  technical  knowledge  a«d 
proficiency  In  carrying  out  HOS  tasks. 


Does  not  display  th* 
ano-ladge/stHl  reouired 
to  p*"'on-  many  jao 

assignments  and  tasks. 

Displays  tne  knowledge/ 
sslll  regu'.refl  to  perform 

...  ,  .  -  ..v  ... 

■0*1  JW  *1* ’g'VW'il* 

tasks  properly.  Out  oay 
need  help  for  harder  tasks 

Displays  the  knowledge/ 
Skill  to  perform  41 1  JOP 
•  iftigiwNrnii  Zit>a» 

propt  r*jr- 

1  2  3  4  @  A  7  I 

E.  PEAfOWAHCE  tWO€*  ADVERSE  CONDITIONS 

Continuing  to  aiecutt  appropriate  soldier  stills 
vnotr  coneat  conditions  or  under  hardship, 
stressful  or  otherwise  difficult  ctrci*ttencdt. 


Haves  frequent  nutates 

Hates  mutates  infra- 

Almost  never  nates  mu* 

in  tom  it  situations  or 

duently  In  comoat  or 

taies  i«  comoat  or 

0th*rw1  St  Stressful 

otnemise  stressful 

othf-wite  stressful 

situations. 

SltUStlO"!. 

situations. 

;  2 

1  i  0“ 

A  7 

Judge  No. 

Flgur*  3.3.  $aupl«  NOS  118  profile  fora 


59 


Name 


Sheet  Ho,  01 


OVERALL  PERFORMANCE  SCORE  SHEET 


Soldier 

Ho. 


Performance  Level 

Demonstrating  Co»-  Technical  Proficiency 
mltraent  to  the  Army  and  Knowledge 


Rank 

Order 


Overall 

Score 


P*rforn*nr*  Scales: 


DEMONSTRATING  COMMITMENT  TO  THE  ARMY 
Maintaining  Army  traditions,  spirit  and  fellowship. 


TECHNICAL  PROFICIENCY  AND  KNOWLEDGE 
Effectiveness  In  applying  technical  Knowledge  and 
proficiency  In  carrying  out  MOS  tasks. 


Does  not  display  the 
kriowl  edge/ski  11  required 
to  perform  many  job 
assignments  and  tasks. 


Displays  the  knowledge/  Displays  the  knowledge/ 
skill  required  to  perform  skill  to  perform  all  job 
most  job  assignments  and  assignments  and  tasks 
tasks  properly,  but  may  properly, 
need  help  for  harder  tasks 


Figure  3.4.  Example  of  Overall  Performance  Score  Sheets. 
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The  world  Is  In  &  period  of  heightened  tensions.  There  is  an 
increasing  probability  that  hostilities  will  break  out  in  Europe. 
Asia,  the  Caribbean.  Latin  Aaerica,  and  Africa.  The  Amy's  mission 
is  to  support  U.S.  treaty  obligations  and  to  help  defend  the 
borders  of  allied  and  friendly  nations.  $o*e  of  the  potential 
eneaies  have  nuclear  and  chemical  capability.  Air  parity  does 
exist  between  allied  forces  and  potential  hostile  nations.  U.S. 
Amy  training  and  other  preparatory  activities  have  been 
substantially  Increased.  Host  combat  and  associated  support  units 
are  participating  in  freouent  field  exercises.  Host  units  are 
being  actively  resupplied. 


Figure  3.5.  Worldwide  increase  in  tensions  scenario. 


The  nethods  were  also  compared  on  three  other  distensions:  judge 
reliability  (intraclass  correlation),  correlation  between  Bean  weights 
assigned  by  the  officers  and  HCOs,  and  the  intercorrelations  among  the  sets 
of  stean  weights  obtained  bv  the  three  aethods  for  all  participants.  These 
statistics  are  shown  in  Table  3.3. 

In  general,  the  conjoint  paired-comparison  method  yielded  the  highest 
intraclass  correlations  for  both  officers  and  HCOs  while  the  conjoint  full- 
profile  aethod  had  the  lowest  values.  The  correlation  between  the  Bean 
officer  and  NCO  weights  obtained  from  the  conjoint  paired-comparisons  aethod 
also  was  the  highest  (t  ■  .91),  while  the  conjoint  full-profile  officer/NCO 
correlation  was  the  lowest  (x.  »  .60).  The  Bean  weights  obtained  fro®  the 
direct  estimation  and  the  conjoint  paired  comparisons  were  highly  correlated 
(r  ■  .93)  while  the  correlations  of  these  weights  with  those  obtained  from 
the  conjoint  full-profile  Bethod  were  quite  low.  On  the  basis  of  these 
results  and  the  participant  aethod  evaluations  described  earlier,  it  was 
decided  to  drop  the  conjoint  full-profile  aethod  from  further  consideration. 

Experiment  3:  Procedure  and  Results 

The  third  experiaent  was  also  conducted  in  the  winter  of  1985,  at  Fort 
Bragg,  using  two  4-hour  workshops.  One  workshop  was  attended  by  seven 
officers,  tne  other  by  eight  NCOS.  The  workshop  participants  were  asked  to 
weight  seven  performance  constructs  for  the  infantry  MOS.  The  seven 
constructs  included  the  five  used  in  the  second  weighting  method  experiaent 
plus  two  additional  ones— avoiding  serious  disciplinary  problems  and 
providing  peer  leadership  and  support. 
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Table  3.2 


Experiment  2:  Kean  Rating!*  of  Weighting  Nethods 
(n  -  IS  officers,  IS  RCOs) 


teltths.ip&itethgd 

Acwtabllit* 

JUS£ 

yaUd  13*  AflTPCSgnt 

Average 

Batina. 

Direct  estimation 

4.30 

5.13 

5.80 

4.77 

5.0G 

Conjoint  paired- 
comparison 

4.23 

4.13 

5.17 

4.50 

4.51 

Conjoint  full- 
profile 

4.27 

3.87 

5.10 

4.23 

4.37 

Significance 

o 

N 

o 

• 

.002 

.048 

NS 

.04 

*  Seven-point  scales  In  which  1  ■ 

Low  and  7  •  High. 

Table  3.3 

Experiment  2:  Agreement  Indexes  for  Weighting  Methods 

Weiqhtirq  Method 

One-Rater 

Officer 

Reliability 

NCQ  ALL 

Correlation 

Off/NCO 

Means 

Intercorrelation 
Full  Paired 
Profile  Cotr.p 

Direct  estimation 

.27 

.24 

.25 

^  « 

.Oi 

•  ^ 

•  A/ 

Conjoint  full- 
profile 

.23 

.01 

.11 

.60 

.15 

Conjoint  paired- 
comparison 

.54 

.32 

.42 

.91 
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Each  participant  used  the  three  different  weighting  methods  described 
below  and  In  the  following  order: 

(1)  Based  on  scores  on  two  constructs,  participants  were  asked  to  rank 
order  21  sets  of  13  Infantrymen  in  order  of  their  overall  perform¬ 
ance.  This  Is  the  same  basic  conjoint  paired-comparison  procedure 
used  in  the  second  experiment.  In  this  case,  however,  in  addition 
to  rank  ordering  the  13  Infantrymen,  the  judges  assigned  perform¬ 
ance  scores  that  reflected  the  soldiers'  relative  overall 
performance. 

(2)  The  participants  were  then  asked  to  rank  order  the  seven  con¬ 
structs,  assign  100  points  to  the  first-ranked  construct,  and  then 
scale  the  other  constructs  accordingly  (the  direct  estimation 
procedure  used  in  Experiments  1  and  2). 

(3)  The  third  method  was  a  variant  of  the  second  and  Incorporated  a 
Delphi  procedure.  Participants  first  indicated  why  they  had 
ranked  and  weighted  performance  factors  as  they  had  in  method  2 
above.  These  reasons  were  passed  around  to  the  other  workshop 
participants;  also  passed  around  were  the  average  and  range  of  the 
weights  given  each  performance  factor  by  the  workshop  participants 
in  method  2.  After  considering  this  feedback  information,  the 
participants  reassigned  weights  to  the  performance  factors  using 
method  2  above.  The  Delphi  procedure  was  then  repeated  once  more. 

The  above  judgments  were  made  in  the  same  context  of  a  worldwide 
Increase  In  tensions  that  was  used  in  Experiment  2.  After  completing  each 
method,  the  participants  rated  the  method  on  the  same  four  7-point  scales 
used  in  the  first  and  second  experiments. 

Table  3.4  presents  the  mean  ratings  given  the  three  weighting  methods 
by  the  15  workshop  participants,  along  with  the  results  of  analysis  of 
variance  tests  of  the  significance  of  the  mean  differences  between  methods. 
The  ratings  for  the  direct  estimation  and  modified  Delphi  methods  were 
generally  higher  than  those  given  the  conjoint  paired-comparison  method. 

It  Is  interesting  to  note  that  while  the  mean  ratings  given  the  direct 
estimation  method  in  Experiments  2  and  3  (>ec  Tables  3.2  sr.d  3.3)  were 
generally  quite  similar,  the  conjoint  paired-comparison  method  generally 
received  lower  ratings  in  Experiment  3  than  in  Experiment  2.  However,  only 
the  mean  acceptability  ratings  for  this  conjoint  method  were  significantly 
different  across  the  two  experiments  (4.23  vs.  3.43). 

The  weighting  methods  used  In  Experiment  3  were  also  compared  on  Inter¬ 
judge  reliability  (intraclass  correlation),  correlation  between  mean  weights 
assigned  by  officers  and  NCOs,  and  intercorrelations  among  the  sets  of  mean 
weights  obtained  by  the  three  methods.  For  the  conjoint  paired-coiparlson 
method,  weights  could  be  derived  by  using  only  the  rank  or  iers  provided  by 
the  judges,  or  by  using  the  overall  performance  scores  assigned  the  sets  of 
13  infantrymen.  Similarly,  for  the  modified  Delphi  method,  weights  could  be 
obtained  from  the  participants'  judgments  after  the  first  round  of  feedback 
or  after  the  second  and  final  round  of  feedback. 
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Tabic  3.4 

Experiment  3:  Keen  Ratings*  of  Weighting  Methods 
(fl  »  7  Officers,  3  NCOs) 


Hating  fctM 

flwpt*W1tn 

&££ 

yiiid.it* 

Aareement 

Average 

Rating.. 

Conjoint  paired 
comparison  (PC) 

3.43 

4.20 

4.60 

3.86 

4.02 

Direct  estimation 

4.21 

5.47 

5.80 

4.57 

4.95 

Modified  Delphi 

4.46 

5.43 

5.93 

4.62 

5.09 

Significance 

NS 

.049 

.010 

NS 

.002 

*  Seven-point  scales  In  which  1  -  lew  and  7  •  High. 


One-rater  reliabilities  were  therefore  calculated  for  five  different 
procedures  of  obtaining  weights  from  the  judgments  provided  by  the  workshop 
participants.  These  reliabilities,  along  with  the  correlations  of  the  mean 
weights  of  the  officer  and  RCO  participants,  are  shown  In  Table  3.5.  The 
correlations  obtained  between  the  five  sets  of  mean  weights  are  shown  in 
Table  3.5.  Also  shown  In  Table  3.6  are  the  Intercorrelations  across  weights 
of  the  five  common  constructs  used  In  Experiments  2  and  3  for  all  the 
methods  used  In  the  two  experiments. 

Several  Inferences  can  be  drawn  from  the  data  presented  In  Tables  3.5 
and  3.6.  First,  there  is  no  evidence  that  the  one-rater  reliabilities  or 
the  correlations  obtained  from  the  officers  and  the  HCOs  are  Improved 
substantially  by  adding  the  requirement  to  provide  overall  performance 
scores  as  well  as  rankings  in  the  conjoint  paired-comparison  method.  Nor 
are  these  agreement  indexes  Improved  by  adolng  one  or  two  rounds  of  Delphi 
feedback  to  the  direct  estimation  method.  Moreover,  the  correlations 
between  weights  obtained  through  the  two  basic  methods  (conjoint  paired- 
comparisons  ranking  and  direct  estimation)  and  the  weights  obtained  through 
their  respective  extensions  (conjoint  paired-comparison  scores  and  Delphi 
rounds  1  and  2}  ranged  from  .96  to  .99. 

Two  other  considerations  led  us  to  decide  that  in  any  future  applica¬ 
tion  of  the  conjoint  paired-comparison  method  the  judges  would  not  be 
required  to  assign  overall  performance  scores  in  addition  to  rank  ordering 
the  sets  of  soldiers.  First,  from  a  practical  point  of  view,  the  require¬ 
ment  to  assign  performance  scores  add?d  about  two  minutes,  on  the  average, 
to  the  amount  of  time  a  judge  takes  to  complete  the  judgment  for  one  set  cf 
13  hypothetical  soldiers. 
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Table  3.5 

Experiment  3:  Agreement  Indexes  for  Weighting  Methods 


Welahtina  Method 

One-Rater  Jtel  lability 

Officer  ICQ  All 

Correlation 

Off/MCO 

Means 

Conjoint  PC  Ranking 

.43 

.27 

.35 

.84 

Conjoint  PC  Scores 

.32 

.20 

.27 

.87 

Direct  EstlMtlon 

- 

.28 

.20 

.25 

.84 

Delphi  Round  1 

.26 

.18 

.22 

.75 

Delphi  Round  2 

.32 

.18 

.24 

.77 

Table  3.6 

Experiments  2  and  3:  Xntercorrelatlons  of  Mean  Weights  Obtained  From  the 
Weighting  Methods  Used  In  Both  Experiments 

Con- 

Con- 

No. 

joint 

Del- 

Del- 

joint 

Of 

PC 

Dir- 

phi 

phi 

Dir-  Full 

Con- 

Rank- 

PC 

act 

Round  Round 

ect  Pro 

Weiahtina  Method 

structs 

inc  Scores  . 

E?t 

1 

2 

_Est  file 

Conjoint  PC  Ranking 

7 

m 

Conjoint  PC  Scores 

7 

.96 

- 

Direct  Estimation 

7 

.73 

.86 

m 

Delphi  Round  1 

7 

.65 

.80 

.96 

- 

Delphi  Round  2 

7 

.64 

.80 

.99 

.97 

- 

Direct  Est  (Exp  2) 

5 

.82 

.91 

.96 

.93 

.93 

- 

Conj  Full-Prof  (Exp  2) 

5 

.12 

.19 

.36 

.44 

.44 

.17 

Conj  PC  (Exp  2) 

5 

.97 

.98 

.87 

.37 

.31 

.93  .15 

65 


The  second  consideration  hts  to  do  with  the  assumption  one  makes  about 
thfc  soldiers  scores  on  the  constructs  that  are  not  being  Iwediately 
coopered  in  the  paired-comparison  protocol.  If  one  assuoes  that  these  other 
construct  scores  ara  all  high,  the  overall  performance  scores  assigned  the 
set  of  soldiers  for  the  pair  of  constructs  being  judged  might  be  different 
than  if  one  assuoes  that  these  scores  are  low,  average,  or  mixed.  The  rank 
orders,  on  the  other  hand,  should  not  be  so  influenced. 

Similar  considerations  led  us  to  decide  not  to  use  the  modified  Delphi 
method  in  addition  to  the  direct  estimation  method. 

The  choice  between  the  direct  estimation  oethod  and  the  conjoint 
paired -cooper Ison  ranking  method  was  not  an  easy  one.  The  direct  estimation 
method  generally  received  higher  evaluation  ratings  in  both  Experiment  2  and 
3  and  would  obviouslv  take  less  time  to  administer  than  the  conjoint  method. 
On  the  other  hand,  the  officer  and  NCO  one-rater  reliabilities  obtained  for 
the  conjoint  method  were  higher  than  for  direct  estimation  in  both  of  the 
experiments.  However,  for  both  the  direct  estimation  and  paired-comparison 
methods  the  correlations  between  the  officer  and  NCO  mean  weights  were  above 
.80  In  both  experiments.  The  correlations  between  the  mean  weights  obtained 
in  Experiment  2  and  those  obtained  In  Experiment  3  were  very  high  for  both 
methods  (.95  for  the  direct  estimation  and  .97  for  the  conjoint  method). 

In  short,  although  each  method  might  have  some  advantages  over  the 
other,  botn  appeared  to  be  sound  methods  of  obtaining  performance  construct 
weights.  We  therefore  decided  to  use  both  methods  to  weight  the  perfcrsar.cc 
constructs  for  MOS  In  the  Project  A  sample. 


OBTAINING  PERFORMANCE  CONSTRUCT  HEIGHTS  FOR  PROJECT  A  NOS 


Procedure 

The  couiponer.l  weighting  judgments  for  Project  A  MOS  were  collected  in  a 
series  of  2-hour  workshops.  Separate  workshops  were  held  for  NCOs  and 
officers  at  each  of  two  posts  for  each  of  20  MOS.  One  of  these  posts  housed 
the  proponent  school  for  the  MGS  and  the  other  housed  field  units  having 
officers  and  NCOs  with  expert  knowledge  of  the  MOS. 

At  each  workshop,  after  a  briefing  on  Project  A,  the  participants  were 
first  given  general  Instructions  which  covered  the  background  and  purpose  of 
the  workshop,  and  descriptions  of  the  performance  components  (constructs) 
and  l.«e  two  methods  (direct  estimation  and  conjoint  paired-comparison 
rank  i nq)  that  would  be  used  to  obtain  weights  for  the  components.  The  two 
scaling  methods  were  then  administered,  always  in  the  case  order.  The 
participants  were  giver,  a  short  break  between  methods. 
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The  components  to  be  weighted  were  the  five  job  performance  criterion 
factors  that  had  been  developed  as  part  of  Project  A's  performance  modeling 
effort  (Campbell,  1986a,  chapter  7).  The  components  were: 

(1)  Task  proficiency:  MOS-speciflc  technical  skills. 

(2)  Task  proficiency:  General  soldiering  skills. 

(3)  Exercise  of  leadership,  effort,  and  self-development. 

(4)  Maintaining  personal  discipline. 

(5)  Military  bearing/appearance  and  physical  fitness. 


Sample  of  Judges 


The  sample  plan  called  for  a  total  of  36  judges  for  each  MOS,  half 
coming  from  field  units  (FORSCOM  and  US.AREUR)  and  half  from  proponent  posts 
(TRADOC).  The  judges  were  to  be  evenly  divided  among  NCOs,  company  grade 
officers,  and  field  grade  officers.  However,  the  target  sample  composition 
was  not  attained  for  every  MOS.  In  some  cases  where  sufficient  numbers  of 
officers  and/or  NCOs  were  not  available,  warrant  officers  who  knew  the  jobs 
well  were  used  in  lieu  of  company  or  field  grade  officers.  Table  3.7  shows 


the  total  sample  of  712  judges  identified  by  MOS,  type  of  post,  and  grade 
level.  Although  sese  individual  MOS  proportions  did  not  meet  the  target, 
overall  the  proportions  of  officers  to  NCOs  and  of  judges  from  field  units 
to  proponent  MOS  posts  were  quite  close  to  the  desired  composition. 


The  Scaling  Methods 


On  the  basis  of  the  results  of  the  earlier  exploratory  experiments,  two 
methods  were  used  to  obtain  importance  weights  for  the  five  performance 
constructs. 


Direct  Estimation.  The  judges  first  rank  ordered  the  five  constructs 
in  terms  of  their  relative  Importance  for  deriving  an  overall  performance 
measure  in  the  given  MOS.  After  assigning  100  points  to  the  most  important 
performance  construct,  the  judges  scaled  the  other  four  constructs  by 
assianing  values  that  reflected  the  importance  they  felt  each  construct 
should  have  In  the  total  effectiveness  score.  The  judges  were  allowed  to 
give  any  relative  weight  f "0«  0  to  100  to  the  other  constructs.  After  they 
initially  assigned  points  to  the  constructs,  the  Judges  were  told  to  review 
the  we  loots  they  had  assigned  and  make  sure  that  they  were  in  correct 
proportion  to  one  another. 

Conjoint  Paired  Comparison.  The  judges  were  given  performance  profiles 
cn  10  sets  of  lb  hypothetical  soldiers  in  the  MOS.  The  15  soldiers  in  any 
on?  \£t  h&d  different  scores  on  two  cf  the  construct*.  The  judgmental  taok 
was  to  rank  the  15  soldiers  in  order  of  their  overall  performance.  Mhe.i  the 
judges  ware  satisfied  with  their  ranking  on  one  set,  they  proceeded  to  the 
ncx.  set  of  15  soldiers,  who  had  different  scores  on  two  other  constructs 
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Table  3.7 

Composition  of  Judging  Simple*  for  Weighting  Project  A  MQS 


Im_9f  UniL 


Field 

Proponent 

_ Total 

m 

HCQ 

Officer 

1  IB 

Infantryman 

17 

6 

19 

6 

36 

12 

12B 

Combat  engineer 

17 

4 

12 

6 

29 

10 

136 

Cannon  Crewman 

6 

6 

21 

6 

27 

12 

I6S 

MANPAOS  Crewman 

11 

6 

11 

5 

22 

11 

19E 

Armor  Crewman 

11 

5 

14 

6 

25 

11 

27E 

TOW/Dragon  Repairer 

6 

16 

S 

16 

11 

31C 

Single  Channel  Radio  Operator 

13 

6 

1? 

6 

25 

12 

51B 

Carpentry /Masonry  Specialist 

4 

6 

2/ 

6 

31 

12 

54E 

Chenical  Operations  Specialist 

20 

14 

20 

14 

55B 

Ammunition  Specialist 

4 

3 

24 

9 

28 

12 

63B 

Light  Wheel  Vehicle  Mechanic 

7 

2 

20 

11 

27 

13 

64C 

Motor  Transport  Operator 

10 

5 

12 

6 

22 

11 

67N 

Utility  Helicopter  Repairer 

12 

1 

17 

12 

29 

13 

71L 

Administrative  Specialist 

13 

6 

9 

7 

22 

13 

76W 

Petroleum  Supply  Specialist 

10 

11 

10 

11 

76Y 

Unit  Supply  Specialist 

15 

5 

8 

5 

23 

10 

91A 

Medical  Specialist 

25 

13 

25 

13 

94B 

Food  Service  Specialist 

12 

7 

8 

4 

20 

11 

95B 

Military  Police 

23 

13 

23 

13 

96B 

Intelligence  Analyst 

11 

6 

11 

6 

Total 

230 

125 

241 

106 

471 

231 

*  in  Addition  to  the  702  officers  and  HCOs  listed  In  this  table,  there  were 
10  judges  whose  grides  were  unknown,  caking  the  total  sample  712. 


The  order  of  presentation  of  the  10  pairs  of  five  constructs  was  governed  by 
the  optinization  procedure  worked  out  by  Ross  (1934).  The  order  of  presen¬ 
tation  of  the  15  soldiers  on  the  score  sheets  was  original ly  randoalzed,  but 
for  ease  In  caking  the  judgments  and  processing  the  data  the  order  reaalned 
the  sace  for  all  10  sets  of  soldiers  for  all  MOS.  However,  the  order  of 
presentation  of  the  10  pairs  of  constructs  was  randoeized  across  MOS. 

In  the  conjoint  method,  the  weights  assigned  by  the  judges  cust  be 
Inferred  fro®  their  rank  ordering  of  the  15  hypothetical  soldiers. 
Presumably,  if  a  judge  consistently  gave  a  higher  rank  to  soldiers  with  high 
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performance  scores  on  one  construct  than  to  soldiers  high  on  the  other 
construct,  then  the  judge  considers  the  first  construct  sore  Important  in 
overall  HOS  performance  than  the  second  construct. 

The  judges  accomplished  the*  two  methods  In  the  order  listed  above.  The 
full  set  of  instructions  and  materials  used  to  collect  the  weighting 
judgments  for  the  Infantryman  HOS  (ilB)  Is  given  In  Appendix  0.  The  judges 
were  given  definitions  of  the  performance  constructs  to  study  before  they 
made  their  judgments.  They  were  asked  tc  assume  that  performance  scores  for 
the  given  HOS  were  available  only  on  the  constructs  given. 

The  judges  were  further  asked  to  assume  that  the  military  context  or 
scenario  In  which  the  soldiers'  performance  was  being  evaluated  was  the 
following: 

The  world  Is  In  a  period  of  heightened  tensions.  There  is  an 
Increasing  probability  that  hostilities  will  break  out  In  Europe, 

Asia,  the  Caribbean,  Latin  America,  and  Africa.  The  Army’s  mission 
is  to  support  U.S.  treaty  obligations  and  to  help  defend  the  borders 
of  allied  and  friendly  nations.  Some  of  the  potential  enemies  have 
nuclear  and  chemical  capability.  Air  parity  does  exist  between  allied 
forces  and  potential  hostile  nations.  tf.S.  Army  training  and  other 
preparatory  activities  have  been  substantially  increased.  Most  combat 
and  associated  support  units  are  participating  1r,  frequent  field 
exercises.  Most  units  are  being  actively  resupplied. 


ANALYSIS  OF  CRITERION  WEIGHTING  DATA 

Data  Transformation 

The  direct  estimation  scaling  method  yielded  weights  on  a  scale  ranging 
fro®  0  to  100.  The  range  and  distribution  of  direct  estimation  weights 
varied  considerably  among  the  judges.  To  better  reflect  the  combined 
judgments  of  the  construct  weights  across  the  judges  for  each  MOS,  the  data 
from  each  judge  were  standardized  prior  to  averaging— a  procedure  that  would 
tend  to  equalize  the  judges'  contributions  to  the  HOS  mean  even  though  they 
may  have  assigned  rather  disparate  sets  of  weights  to  the  constructs. 

To  preserve  the  relative  size  of  the  weights  that  each  judge  had 
assigned  the  constructs,  each  judge's  weights  were  transformed  by  multiply¬ 
ing  them  by  a  constant  (the  ratio,  100/sum  of  the  judge's  weights).  This 
caused  the  five  construct  weights  of  each  judge  to  sum  to  100,  but  did  not 
change  the  relative  values  of  the  judge's  weights.  Consequently,  the 
average  of  the  five  construct  weights  of  all  judges  was  set  at  20.0,  and  the 
average  of  the  five  weights  for  any  group  of  judges  within  and  across  MOS 
was  also  set  at  20.0.  The  mean  weight  of  a  given  construct  obtained  by 
averaging  the  judges'  Individual  weights  could,  of  course,  be  different 
from  20. 

For  the  conjoint  method,  the  data  from  each  judge  wore  scaled  using  a 
method  developed  by  Comrey  (1950)  which  Is  described  in  Torgerson  (1958;. 


Essentially,  the  multiple  regression  equation  predicting  the  Judge's  rank 
orders  of  the  two  performance  construct  scores  of  the  IS  hypothetical 
soldiers  was  first  obtained  for  each  of  the  10  sets  of  soldiers.  The  ratio 
of  the  two  regression  weights  for  each  pair  of  constructs  then  became  the 
basic  data  entering  Into  the  scaling  procedure.  Since  the  correlation 
between  the  two  construct  scores  of  the  15  hypothetical  soldiers  on  each 
performance  rating  sheet  was  specified  to  be  zero,  the  ratio  of  the 
regression  weights  is  directly  proportional  to  the  correlation  of  each  set 
of  construct  scores  with  the  Judge's  rank  order  of  the  soldiers.  (The  means 
and  standard  deviations  of  the  construct  scores  were  equal  for  all 
constructs.) 

The  scaling  procedure  employs  a  least  squares  solution  to  obtain  a  set 
of  weights  that  best  fit  the  observed  ratios.  The  resultant  weights  are  so 
scaled  that  their  geometric  mean  is  1.0.  To  facilitate  comparing  the 
conjoint  weights  with  those  obtained  by  the  direct  estimation  method,  the 
conjoint  weights  for  each  Judge  were  also  linearly  transformed  so  that  their 
sum  was  equal  to  100  and  their  average  equal  to  20.0. 

One  reason  for  effecting  the  transformation  concerned  the  practical 
application  of  the  weights  to  the  construct  scores.  The  final  Intent  Is  to 
apply  a  set  of  weights  to  the  construct  variance/covariance  matrix  such  that 
the  covariance  of  each  construct  with  the  composite  total  score  Is 
equivalent  to  the  construct  weight  obtained  from  the  Judges.  In  other 
words,  the  contribution  to  the  total  M0S  performance  variance  of  each 
construct  would  be  directly  proportional  to  Its  weight.  A  separate 
algorithm  will  be  used  to  calculate  the  weights  that,  when  applied  to  the 
variance/covariance  matrix,  yield  the  desired  (the  obtained  scaled)  weights 
or  contributions  to  the  total  composite  variance. 

Examination  of  Hissing  Data 

For  the  conjoint  scaling  method,  73  of  the  Judges  either  had  failed  to 
complete  the  entire  judgmental  sequence  or  had  recorded  judgments  that  were 
Inconsistent  with  the  assumptions  of  the  scaling  method  Involved.  For 
example,  a  judge  may  have  completed  all  the  performance  score  sheets,  but 
one  or  more  of  the  10  resultant  regression  equations  had  constructs  with  a 
positive  weight.  This  would  mean  that  the  higher  a  Judge  rank  ordered  the 
15  hypothetical  soldiers  on  the  given  score  sheet,  the  lower  were  the 
soldiers'  scores  on  one  of  the  constructs.  However,  the  scaling  method 
employed  (see  Torgerson,  op.  clt.)  required  that  both  weights  have  the  same 
sign  and  that  a  full  set  of  weight  ratios  be  available.  Consequently, 
either  the  conjoint  protocols  with  missing  or  positive  weights  could  be 
eliminated  or  the  missing  weight  ratios  could  be  Imputed  by  an  appropriate 
estimation  technique.  As  can  be  seen  in  Table  3.8,  proportionately  more 
NCOs  than  officers  had  one  or  more  problems  of  this  nature  in  their  conjoint 
protocols. 

In  order  to  keep  at  least  some  of  these  judges,  the  missing  weight 
ratios  for  those  Judges  who  had  only  one  conjoint  performance  score  sheet 
uncompleted  or  who  had  only  one  pair  of  weights  of  opposite  sign  were 
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Tibia  3.8 

Conjoint  Hethod  Hissing  or  Invalid  Data  by  Grade 


Grade 


Amount  _ol_MlFiiiiQ_D*U 

BUP 

Pflton 

Musa 

Mfil 

1  data  element  missing  or  invalid 

16 

19 

0 

35 

2  or  more  data  elements  missing  or  invalid 

22 

16 

0 

38 

Ho  data  missing 

193 

436 

10 

639 

Total 

231 

471 

JO 

712 

a  Proportion  of  HCOs  having  missing  data  Is  higher  than  officers 
(  2  -  13.53,  df  ■  2,  significant  at  .01  level). 


estimated  by  the  technique  described  below.  Judges  with  two  or  aore 
proDiems  in  ineir  conjoint  uaU  {33}  were  dropped  free  the  conjoint  date 
set. 


The  imputed  estimates  of  the  weight  ratios  were  obtained  by  first 
correlating  the  judge's  nonmissing  ratios  with  the  ratios  of  other  judges 
within  the  MOS  who  had  no  missing  data,  and  then  computing  a  stepwise 
multiple  regression  equation  to  predict  the  missing  ratios,  ho  equation 
could  be  computed  for  seven  of  the  35  judges  with  one  key  data  element 
missing  because  no  other  judge  had  values  sufficiently  correlated  with  these 
judges'  ratios;  these  seven  were  dropped  from  the  analysis. 

The  28  judges  for  whom  we  imputed  the  missing  regression  weight  ratios 
were  then  compared  with  the  remaining  judges  on  two  indexes: 

(1)  The  correlation  between  each  judge's  set  of  weights  produced  by 
the  direct  estimation  and  the  conjoint  scaling  methods. 

(2)  The  consistency  with  which  a  judge  rank  ordered  the  15 
hypothetical  soldiers  on  the  basis  of  their  construct  scores. 

For  example,  If  a  judge  ranked  a  hypothetical  soldier  with 
sre’es  of  "5"  and  *3*  on  two  performance  scales  lower  than 
another  soldier  with  scores  of  "S"  and  “S",  the  judge  would  be 
giving  a  higher  rank  to  a  poorer  performing  soldier.  In  the  set 
of  10  conjoint  performance  score  sheets,  a  maximum  of  630  such 
rank  order  inversions  was  possible. 
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The  judges  with  Imputed  conjoirt  scale  values  had  somewhat  lower  direct 
estimation/conjoint  correlations  between  their  scale  values  than  did  judges 
with  coaplete  data  and  also  had  Bore  Inversions  in  the  rank  orders  they 
assigned  to  hypothetical  soldiers  listed  on  the  conjoint  performance  score 
sheets  (see  Table  3.9).  Consequently,  these  judges  were  also  dropped  fro* 
the  analyses  of  the  conjoint  data. 

Analyses  of  Outliers 

As  was  seen  in  Table  3.9,  a  number  of  the  remaining  639  judges  had  a 
large  number  of  inversions  In  the  rank  orders  the>  assigned  the  15 
hypothetical  soldiers.  A  within-MOS  analysis  was  conducted  In  which  judges 
with  the  highest  number  of  Inversions  were  progressively  dropped  fro*  the 
sample.  After  each  successive  judge  was  dropped,  the  average  l-rater  and 
rater  Intracless  correlations  or  reliability  coefficients  for  the  remaining 
pool  of  judges  were  calculated.  The  average  n-rater  reliabilities  across 
the  20  MOS  proved  to  be  highest  when  the  two  judges  with  the  largest  number 
of  Inversions  wore  eliminated. 

Consequently,  the  two  judges  in  each  MOS  who  had  the  hiahest  number  of 
Inversions  were  dropped,  provided  that  they  had  at  least  30  Inversions  in 
their  protocol.  In  addition,  any  judge  with  90  or  more  inversions  was 
dropped  even  if  this  meant  that  more  than  two  judges  were  eliBinated  for  a 
given  MOS.  Altogether,  40  judges  were  dropped.  The  average  1-rater  and  £- 
rater  reliabilities  across  the  20  MOS  were  .221  and  .879,  respectively, 
before  the  40  judges  were  dropped,  and  .236  and  .881  after  they  were 
el ieinated ; 

Table  3.9 

Frequency  Distribution  of  Inversions  Made  by  Judges  With  Imputed  and 
Hot-Imputed  Conjoint  Ratios* 


No.  of  Inversions 

Jt9.tJap.alsd 

AH  Jydqja 

0 

4 

75 

75 

1-19 

/ 

412 

416 

Z0-39 

8 

96 

105 

40-59 

3 

29 

37 

60-79 

3 

10 

13 

80-99 

1 

7 

10 

100-119 

2 

3 

120-139 

3 

3 

140-159 

2 

2 

160-179 

2 

2 

180-199 

— 

_ I 

Total 

28 

639 

667 

*  Median  test  results:  2  -  25.28  (significant  at  .001  level). 
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.While  these  gains  In  reliability  for  the  conjoint  judgments  were  not 
large,  direct  estimation  reliabilities  also  Improved  with  use  of  the  reduced 
sample.  The  1-rater  and  ji-rater  reliabilities  for  the  direct  estimation 
method  averaged  .186  and  .854  when  all  the  712  judges  were  used  but  rose  to 
.223  and  .663  with  the  reduced  sample  of  599.  Dropping  judges  who  apparent¬ 
ly  had  not  accomplished  the  conjoint  procedure  carefully  helped  1iq>rove  the 
reliability  of  tne  weights  assigned  the  constructs  under  both  methods.  All 
remaining  analyses  were  carried  out  on  the  reduced  sample  of  599  judges. 


RESULTS 


Inter judge  Reliability  and  Intermethod  Agreement 

The  Intraclass  reliabilities  of  the  direct  estimation  and  conjoint 
weights  are  shown  In  Table  3.10  by  grade  and  MOS.  The  average  KCO  1-rater 
and  £-rater  reliabilities  for  the  direct  estimation  and  conjoint  scaling 
methods  were  .132/. 425  and  .153/. 509  respectively.  The  corresponding  values 
for  officers  were  .278/. 864  and  .287/. 867. 


As  shown  In  Table  3.11,  for  officer  judgments  the  correlations  across 
the  20  MOS  of  the  average  weights  derived  from  the  direct  estimation  and 
conjoint  scaling  methods  ranged  from  .836  to  .996;  the  average  Intermethod 
agreement  was  .951.  The  corresponding  range  for  the  NCOs  was  .017  to  .922 
and  their  average  MOS  Intermethod  agreement  was  .653.  These  intermethod 
results  in  part  reflect  that  lower  1-rater  reliabilities  were  obtained  for 
ndcr  both  methods  and  also  that  there  were  fewer  MCO  judges. 
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Another  factor  that  may  have  had  some  effect  was  the  greater 
homogeneity  of  the  weights  that  the  HCOs  assigned  the  five  constructs.  The 
average  of  the  standard  deviations  of  the  weights  assigned  by  the  Individual 
NCOs  across  both  methods  was  6.43,  while  the  corresponding  officer  average 
standard  deviation  was  7.69  (see  Table  3.12).  The  difference  between  these 
means  was  statistically  significant  (£>  .001). 

Comparison  of  Direct  Estimation  and  Conjoint  Scaling  Methods 

To  decide  whether  the  final  set  of  weights  should  be  obtained  from  the 
direct  estimation  or  the  conjoint  method,  the  two  sets  of  weights  were 
compared  on  several  indexes.  Though  in  general  the  differences  were  slight, 
they  all  favored  the  conjoint  method.  The  1-rater  and  £-rater  intraclass 
reliabilities  for  the  combined  group  of  officers  and  NCOS  tended  to  be 
slightly  higher  for  the  conjoint  method  across  the  20  MOS  (see  Table  3.13). 
While  the  differences  between  the  reliabilities  for  the  two  scaling  methods 
were  slightly  greater  for  the  NCOs  than  for  the  officers,  the  difference 
favored  the  conjoint  method  In  each  case. 


In  general,  the  weights  assigned  the  constructs  by  the  NCOs  correlated 
higher  with  those  assigned  by  the  officers  when  the  conjoint  scaling  method 
was  used  (Table  3.11).  Across  the  20  MOS,  the  correlations  between  the  HC0/ 
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Table  3.11 


Correlations  Between  Construct  Weights  by  Method,  Grade,  and  NOS 


Direct 


HOS 

Offtesra 

Estipatlon 

&£joM 

11B 

.211 

.963 

.931 

.590 

.546 

12B 

.897 

.973 

.973 

.665 

.631 

13B 

.858 

.980 

.983 

.607 

.648 

16S 

.571 

.957 

.932 

.707 

-.776 

19E 

.891 

.935 

.944 

.695 

.888 

27E 

.691 

.836 

.783 

.820 

.908 

31C 

.822 

.989 

.986 

.649 

.857 

51B 

.085 

.933 

.955 

.515 

.719 

54E 

.921 

.965 

.980 

.563 

.670 

55B 

.737 

.866 

.939 

-.107 

.100 

63B 

.551 

.968 

.987 

.615 

.837 

64C 

.017 

.985 

.961 

.796 

.364 

67N 

.922 

.996 

.991 

.819 

.965 

71L 

.772 

.966 

.962 

.919 

.968 

76W 

.575 

.946 

.956 

-.379 

.451 

76Y 

.780 

.919 

.942 

.677 

.965 

91A 

.805 

.975 

.964 

.696 

.940 

94  B 

.685 

.984 

.981 

.556 

.810 

95B 

.731 

.884 

.918 

.773 

.737 

96B 

.542 

.947 

.924 

.95 8 

.329 

Average 

.653 

.951 

.950 

.617 

.705 

Tibi®  34? 

Average  Standard  Deviation  of  the  Construct  Weights  Assigned  by  the  Judges 


Direct 

Estlw.Upn 

Conjoint 

Total 

NCO 

168 

4.53 

8.33 

6.43 

Officer 

421 

LSI 

LSi 

UtS 

Total 

592 

5.46 

9.20 

7.33 
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Table  3.13 


Intraclcss  Reliabilities  of  Direct  Estimation  end 
Conjoint  Weights  by  NOS  (Officers  end  NCOs  Combined) 


m 

~£L 

iTlflfr 

ne 

37 

.261 

128 

35 

.273 

138 

34 

.249 

16S 

28 

.359 

19E 

30 

.310 

271 

21 

.164 

31C 

31 

.202 

518 

37 

.136 

54E 

31 

.147 

55B 

33 

.247 

63S 

33 

.270 

64C 

29 

.208 

67N 

37 

.315 

71L 

31 

.205 

76k 

«  # 

*•» 

•  uc/ 

76Y 

29 

.233 

91A 

36 

.247 

948 

28 

.187 

95B 

31 

.194 

96B 

14 

.242 

Average 

.233 

nrnittr 

i.-iatcc 

n-ratcr 

.929 

.236 

.920 

.929 

.324 

.944 

.918 

.356 

.949 

.940 

.307 

.925 

.928 

.362 

.944 

.804 

.237 

.867 

.887 

.262 

.917 

.853 

.157 

.873 

.842 

.160 

.855 

.915 

.188 

.884 

.924 

.261 

.921 

.884 

.123 

.803 

.945 

.302 

.941 

.889 

.207 

.890 

n  ok 

no7 

»  »*«  ■ 

.898 

.173 

.858 

.922 

.295 

.938 

.865 

.191 

.869 

.882 

.256 

.914 

.818 

.234 

.801 

.863 

.236 

.881 

officer  mean  conjoint  weights  ranged  from  ,100  to  .968  with  an  average  of 
.705.  The  corresponding  range  for  the  direct  estimation  weights  was  -.379 
to  .958,  with  an  average  of  .617. 

The  slight  overall  psychometric  superiority  of  the  conjoint  weights  may 
be  due  in  part  to  the  larger  discriminate lity  of  the  weights  obtained  fro® 
the  conjoint  method.  The  average  standard  deviation  across  all  judges  of 
the  weights  assigned  by  the  conjoint  method  was  9.20;  the  corresponding 
average  was  5.46  for  the  direct  estimation  method  (Table  3.12). 

Considering  the  above  findings,  the  decision  was  made  to  favor  the 
weights  derived  from  the  conjoint  scaling  method  in  combining  the  individual 
construct  scores  into  an  overall  composite  measure  of  performance. 
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The  sean  weights  obtained  through  the  conjoint  scaling  method  are  shown  In 
Table  3.14  broken  down  by  construct,  grade,  and  HOS.  It  should  be  borne  In 
■Ind  that  the  weights  are  based  on  cooperative  judgments  of  the  constructs 
within  each  HOS  and  should  ufit  be  used  for  comparisons  of  importance  across 
HOS.  It  is,  however,  Interesting  to  note  whether  the  relative  pattern  of 
weights  differs  across  HOS  and  whether  some  constructs  arc  fairly 
consistently  given  relatively  higher  weights  than  others. 

To  explore  differences  In  the  relative  pattern  of  weiohts,  an  analysis  of 
variance  of  the  conjoint  weights  was  conducted  to  test  for  mean  construct 
differences  and  for  any  significant  interactions  with  grade  and  HOS.  The 
analysis  also  compared  the  mean  weights  assigned  by  judges  drawn  from  HOS 
proponent  posts  with  those  of  judges  from  USAREUR  and  FORSCOM  posts.  The 
means  for  prade  (officer  vs-  NCO),  type  of  unit  (field  vs.  proponent),  and 
HOS  main  effect  were  set  at  20.0  by  the  scaling  method  and  hence  were  not  a 

_ - i  -  A  i n  t  e  _  -  x  I 1  x  ^  .  /  xL_  _ _  1  1  . . .  1 .  .A  i  ^ 
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of  variance.  The  construct  means  were  significantly  different.  The 
Interactions  of  constructs  with  grade  and  HOS  were  also  highly  significant, 
indicating  that  the  relative  weights  were  different  for  officers  In 
comparison  with  NCOS  and  were  also  different  across  HOS.  Finally,  there  was 
a  significant  three-way  interaction  among  the  constructs,  HOS,  and  type  of 
judge  (field  vs.  proponent  post). 

LAOwi *«t  ion  c?  the  construct  means  In  Table  3.14  shows  that  for  all  20 
HOS,  Hilitary  Bearing/Physical  Fitness  received  the  lowest  relative  weight. 
In  13  of  the  20  HOS,  Core  Technical  Skills  received  the  highest  relative 
weight,  while  the  Exercising  Leadership  construct  was  second  overall.  The 
Leadership  component  received  the  highest  relative  weight  in  6  of  the  20 
HOS.  For  the  most  part,  the  HOS  Skills  construct  received  the  highest 
weight  for  the  technical  HOS  in  the  sample  and  the  Exercising  Leadership 
construct  received  the  highest  weight  for  the  combat  HOS  (the  job  of  Armor 
Crewman  is  a  notable  exception).  The  General  Skills  construct  received  the 
highest  weight  for  only  one  M0S,  Military  Police  (95B).  These  M0S 
differences  in  the  constructs  receiving  the  highest  weights  undoubtedly 
contributed  to  the  significant  Construct  by  ilQS  interaction. 

Significant  mean  differences  between  the  weights  assigned  by  officers 
and  NCOS  were  found  for  two  constructs:  Officers  gave  significantly  higher 
relative  weights  to  the  Exercising  Leadership  construct  than  did  NCOs,  while 
NCOs  gave  higher  weights  to  the  Military  Bearing/Physical  Fitness  construct 
than  did  officers.  The  NCOs  may  have  been  giving  relatively  more  weigSit  to 
aspects  of  first-tour  soldiers1  performance  that  were  of  more  immediate 
concern  to  them.  Although  the  mean  differences  were  only  significantly 
different  at.  the  .10  level,  the  NCOs  gave  the  Personal  Discipline  construct 
weights  that  were  higher  on  the  average  than  those  assigned  by  the  officers. 
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Toward  the  end  of  the  data  collection,  a  field  experiment  was  run  to 
determine  whether  a  change  in  scenario  would  affect  the  weighting  judgments. 
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Table  3.15 

Results  of  Overall  Analysis 

of  Variance  of  Conjoint  Weights 

Sum  of 

Mean 

¥ 

Source  of  Variation 

4f. 

Savers 

Sfld.pr.fi 

lHai 

M. 

Between  Subjects® 

Grade 

1 

0.0 

MOS 

19 

0.0 

Type  of  Unit 

1 

0.0 

Grade  X  MOS 

19 

0.0 

Grade  X  Type 

1 

0.0 

MOS  X  Type 

14 

0.0 

Grade  X  MOS  X  Type 

13 

0.0 

Error 

523 

0.0 

Within  Subjects 

Constructs 

4 

52604.8 

13151.2 

161.9 

.0000 

Constructs  X  Grade 

4 

2694.3 

673.5 

8.3 

.0001 

Constructs  X  MOS 

76 

14133.1 

186.0 

2.3 

.0001 

Constructs  X  Type 

4 

432.3 

108,1 

1.3 

.2562 

Constructs  X  H05  X  Grade 

•m  r 

/o 

arr\*\* s  r 
o ? ju.a 

m  n 

9A*L 

1  \ 

4.  *•  A 

*  t 

Constructs  X  Grade  X  Type 

4 

6C.1 

15.0 

.2 

.9464 

Constructs  X  MOS  X  Type 

56 

6373.9 

113. B 

1.4 

.0276 

C  X  G  X  MOS  X  T 

52 

3275.6 

63.0 

.8 

.8781 

Error 

2092 

169947.7 

aThe  between-subjects  sum  of  squares  is  equal  to  zero  since  the  Heights  for 
all  subjects  sunmed  to  100. 

Using  the  direct  estimation  scaling  method,  officers  and  NCOs  in  13  K05 
judged  the  relative  weight  of  the  five  performance  constructs  under  a 
wartime  and  a  peacetime  scenario,  after  they  had  completed  judging  the 
constructs  under  the  heightened  tensions  scenario,  using  both  the  direct 
estimation  and  conjoint  methods.  The  two  additional  scenarios  were  as 
described  In  Figure  3.2. 

An  analysis  of  variance  was  conducted  on  the  data  fro®  139  officers  and 
37  MCOs  who  judged  the  five  constructs  under  ell  three  scenarios.  Of 
particular  interest  was  whether  the  within-subject  Scenario  by  Construct 
interaction  ter*  was  significant,  since  that  would  indicate  whether  the 
judges  changed  the  relative  weights  assigned  one  or  more  constructs  as  & 
function  of  the  scenario. 


•  «*■ 
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The  Scenario  by  Construct  Interaction  was  significant,  and  separate 
analyses  of  variance  were  conducted  for  each  construct  to  help  determine 
which  construct  weights  were  influenced  the  Host  by  the  different  scenarios. 
These  analyses  indicated  that  the  means  of  the  MOS  Skills,  6eneral  Skills, 
and  Military  Bearing/Physical  Fitness  construct  weights  were  significantly 
different  across  the  scenarios  (see  Table  3.16).  The  Military  Bearing/ 
Physical  Fitness  construct  received  relatively  eore  weight  under  the 
peacetime  scenario  than  It  did  under  the  heightened  tensions  and  wartime 
scenarios.  The  General  Skills  construct,  on  the  other  hand,  received 
relatively  acre  weight  under  the  wartime  scenario  than  under  the  heightened 
tensions  and  peacetime  scenarios,  while  the  MOS  Skills  construct  received 
Its  highest  weights  under  the  heightened  tensions  scenario. 

Although  these  scenario  differences  were  statistically  significant,  the 
actual  »ean  differences  were  quite  small  and  the  rank  ordering  of  the  five 
components  did  not  change  across  scenarios.  Also,  the  correlations  between 
the  weights  assigned  under  the  three  scenarios  averaged  about  .85  across  the 
13  MOS.  With  weights  correlated  that  highly,  overall  performance  composites 
obtained  through  applying  the  separate  sets  of  scenario  weights  to  construct 
scores  would  most  likely  correlate  between  .95  and  .99.  As  a  consequence, 
we  can  predict  with  certainty  that  alternative  criterion  composites  based  on 
different  scenario  weight;  will  jjot  yield  different  predictor  equations. 

It  is  interesting  to  note  that  there  was  tore  discrlminabllity  in  the 
weights  assigned  the  constructs  within  MOS  under  the  heightened  tensions 
scenario  than  under  the  peacetime  and  wartime  scenarios.  When  the  standard 
deviations  of  the  mean  (fer  ji  judges)  construct  weights  for  each  MGS  were 
averaged  across  MOS,  the  means  were  5.33,  4.76,  and  4.80  respectively  and 
there  mean  differences  were  significantly  different  at  the  .001  level.  The 
reliabilities  of  the  weights  assigned  under  the  heightened  tensions  scenario 
were  also  higher.  Across  the  13  MOS  the  average  1-rat.er  reliability  for  the 
heightened  tensions  scenario  was  .224.  The  corresponding  average 
reliabilities  for  the  peacetime  and  wartime  scenarios  were  .137  and  .202. 


Table  3.16 

kiean  Weights  of  Constructs  by  Scenario  (Based  en  Data  From  !3  MOS) 


■C9Hsiru£t 

t£«ceiifi£ 

Heightened 
.  Tension?. 

foiiipe 

MOS  Skills6 

21.6 

22.3 

21.7 

6eneral  Skills0 

19.9 

20.4 

21.3 

Exercising  Leadership 

21.4 

21.8 

21.5 

Personal  Discipline 

19.9 

19.6 

19.9 

Military  Bearing® 

17.1 

15.8 

15.7 

Construct  means  significantly  different  across  scenarios  at  .05  level. 
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SZSCUSSION  AMD  CONaUSZOMS 


The  five  Project  A  performance  constructs  received  significantly 
different  patterns  of  weights  in  different  MOS  (e.g.,  see  Table  3- 14)  and 
the  different  groups  of  experts  agreed,  In  general,  on  the  relative  ranking 
of  the  weights.  For  example,  the  Leadership/Effort  construct  tends  to  be 
rated  highest  among  the  combat  MOS. 

Multiple  judges  per  MOS,  about  30  on  the  average,  produced  n-rater 
reliabilities  that  are  quite  respectable  (above  .85  for  most  MOS).  The  high 
intermethod  correlations  (about  .95  on  the  average)  between  the  construct 
weights  obtained  by  the  direct  estimation  and  conjoint  methods  fcr  the 
separate  MOS  further  document  the  reliability  of  the  means  of  the  scaled 
weights. 

That  different  groups  of  judges  may  provide  somewhat  different  MOS 
weights  can  be  seen  in  the  correlations  between  the  officer  and  KCO  weights, 
of  .617  and  .705  for  the  direct  estimate  and  conjoint  methods,  respectively. 
The  NCOs  tended  to  give  relatively  higher  weights  to  the  Military  Bearing/ 
Physical  Fitness  construct,  while  the  officers  attached  more  Importance  to 
the  Leadership/Effort  construct.  The  NCOs  could  have  been  reacting  more  to 
the  every- day  problems  of  handling  first-tour  soldiers,  while  the  officers 
could  have  been  more  concerned  with  performance  characteristics  required 
most  under  near  or  actual  wartime  conditions.  The  pattern  of  results 
obtained  when  the  weights  were  evaluated  under  wartime  and  peacetime 
scenarios  in  part  supports  this  hypothesis. 

Though  there  were  statistically  significant  differences  in  the  mean 
weights  assigned  under  the  three  scenarios,  the  very  small  differences  will 
have  little  impact  on  the  relative  ranking  of  soldiers  on  the  overall 
performance  composites  for  an  MOS.  A  more  critical  question  is  how  much 
impact  will  the  weights  themselves  have  on  recommended  job  assignments  in  an 
optimal  selection  and  classification  system?  Would  the  same  assignment 
recommendations  be  made  were  all  weighted  equally?  Would  a  different  set  of 
predictors  be  selected  using  a  weighted  composite  for  validation  than  would 
have  been  selected  If  the  constructs  had  been  weighted  equally? 

The  answers  to  these  questions  obviously  depend  not  only  on  the  set  cf 
weights  used  but  on  such  factors  as  the  intercorreiatlons  among  the 
construct  performance  scores,  the  validity  of  the  predictor  battery,  the 
amount  of  differential  prediction  it  affords  across  Army  jobs,  the  MOS 
selection  standards  in  effect,  and  the  assignment  algorithms  employed. 

Applying  construct  weights  to  performance  scores  to  obtain  a  composite 
score  involved  the  difference  between  what  might  be  called  nominal  and 
operative  weights.  In  nominal  weighting  the  raw  score  on  each  component  Is 
multiplied  by  the  $ME-derived  weight  for  the  component  and  scores  are  then 
added  across  components  to  get  the  total  criterion  composite  score. 

However,  a  component's  operative  weight  for  determining  the  overall  ranking 
of  people  on  the  total  composite  is  also  a  function  of  its  variance  and  its 
covariances  with  the  other  components.  Components  with  higher  variances 
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carry  more  weight,  and  differential  weights  have  less  differential  effect  as 
covariances  become  higher. 

The  alternatives  to  the  nominal  score  process  of  cross-multiplying  SME 
weights  with  raw  component  scores  are  to  (a)  standardize  the  component 
scores  to  control  for  variance  differences;  (b)  “assign"  the  total  composite 
variance  (which  Is  the  sub  of  all  component  variances  and  covariances)  to 
components  by  adding  a  particular  component's  variance  to  its  covariances 
with  each  of  the  other  components,  and  choosing  weights  for  the  components 
that  will  make  their  proportion  of  the  total  variance  equal  to  the  SME- 
determined  weight;  or  (c)  reconstitute  the  component  scores  as  orthogonal 
vectors  and  assign  weights  to  these  variables. 

The  most  straightforward  method  would  be  to  apply  the  SME  weights  to 
standardized  component  scores  and  let  the  reality  of  the  Intercorrelations 
among  the  components  have  their  Influence.  However,  the  most  Informative 
way  to  address  these  questions,  and  the  other  Issues  discussed  above,  is 
through  a  series  of  sensitivity  analyses  that  portray  the  effects  of  these 
parameter  on  selection  and  classification  validity. 

To  the  extent  that  the  differential  weights  described  here  enhance  the 
overall  Army  selection  and  classification  process,  the  time  and  effort  that 
have  gone  into  developing  them  will  be  more  than  worthwhile.  However,  even 
if  the  weights'  effect  on  the  selection  and  classification  process  proves 
minimal,  we  will  have  developed  defensible  performance  composites  for  the 
Project  A  sample  MOS  to  use  as  overall  criterion  measures  In  validating  the 
ASVAB  and  other  selection  instruments  and  procedures. 
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Chapter  4 

SCALING  THE  UTILITY  Of  006  PERFORMANCE1 


INTRODUCTION 


This  chapter  describes  the  Project  A  research  to  determine  the  relative 
utility  to  the  Army  of  different  levels  of  performance  In  entry-level  HOS. 
The  aaln  purpose  of  the  utility  measurement  component  In  Project  A  Is  to 
provide  Information  that  will  aid  dec Is Ion-Bakers  In  maximizing  the  payoff 
to  the  Army  of  luproved  selection  and  classification  procedures. 

Two  major  Issues  In  developing  and  evaluating  a  personnel  selection  or 
classification  system  are  how  to  maximize  the  oain  to  the  organization  from 
using  the  system,  and  how  to  assess  the  net  gain  to  the  organization  fro* 
using  the  new  systea  versus  not  using  It.  To  answer  such  questions,  at 
least  three  major  elements  are  needed; 

e  A  wodel  that  portrays  the  relevant  parameters  In  the  decision- 
making  process  and  specifies  how  they  are  Interrelated. 

e  A  metric  that  can  be  used  to  represent  the  value  of  the  outcomes 
that  result  froc  a  particular  course  of  action. 

e  A  Method  for  estimating  the  paraaeters  of  the  Model  In  the 
appropriate  metric. 

We  know  a  fair  amount  about  modeling  personnel  selection  decisions 
(e.g.,  Cronbach  i  Gleser,  1965)  and  somewhat  less,  but  still  quite  a  bit, 
about  modeling  personnel  classification  decisions  (e.g.,  Roulon,  Tledeoan, 
Tatsuoka,  &  Langmuir,  1967).  A  great  deal  of  effort  by  psychometriclans  and 
Industrial  psychologists  has  been  put  Into  developing  and  refining  such 
models  (cf.  Casein,  1982a).  He  are  much  less  clear  as  to  the  metric  in 
which  the  outcomes  of  a  personnel  selection  or  classification  decision 
should  be  expressed. 


The  Utility  Issue  in  Industrial  j_sycho_ioflY 

Although  the  steps  described  below  have  not  occurred  In  a  perfect 
chronological  order,  the  progression  of  attempts  by  psychometricians  and 
personnel  researchers  to  portray  the  benefits  of  acumen  In  selection  and 
classification  has  been  somethi/ig  like  the  following. 

The  validity  coefficient,  in  the  form  of  the  product  moment  correlation 
between  a  predictor  composite  and  a  criterion  composite,  Is  the  classic 
method  by  which  the  value  of  a  selection  program  is  represented.  However, 


*Th1s  chapter  Is  based  on  Sadacca  and  Campbell  (1987)  and  Sadacca, 
White,  Campbell,  01  Fazio,  and  Schultz  (In  preparation). 
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to  Interpret.  Early  on,  a  number  of  transformations,  such  as  the 
coefficient  of  determination  C^xy)*  the  index  of  forecasting  efficiency 
(l-  |/l-£  xy)  <  the  standard  irror  of  prediction  (Sv  |  were 

suggested  ana  found  wanting.  They  still  depended  very  heavily  on  the 
correlation  coefficient  itself,  and  cannot  be  Interpreted  directly  In  terms 
of  benefits  from  decision  making. 


A  more  useful  kind  of  transformation  is  represented  by  the  various  ways 
of  using  the  bivariate  distribution  to  construe^  decision  tables.  The 
Taylor-Russell  tables  (Taylor  A  Russell,  1939)  are  an  example.  With  these 
transformations,  the  metric  becomes  the  proportion  of  correct  predictions 
that  are  made  by  one  selection  method  versus  another.  One  benefit  of 
looking  at  selection  payoff  1r.  terms  of  decision  accuracy  Is  that  it 
Illustrates  quite  clearly  how  even  a  small  relationship  between  predictor 
and  criterion  can  produce  significant  gains  In  the  number  of  successful 
people  selected  if  the  selection  ratio  is  very  low  and/or  the  variability  In 
performance  is  high  (e.g.,  base  rate  for  success/failure  ■  .50).  However, 
to  express  the  value  of  selection  in  these  terms,  the  organization  must 
define  specific  criterion  categories  (e.g.,  successful  versus  unsuccessful 
performance)  and  nst  view  all  the  outcomes  in  a  particular  category  as 
being  equally  valuable. 

A  new  dimension  was  added  by  the  classic  work  of  Brogden  (1946,  1949), 
who  showed  that  If  both  the  predictor  and  the  criterion  measures  had 
Interval  properties  and  if  the  relationship  between  them  was  linear,  then 
the  correlation  coefficient  Is  linearly  related  to  the  gain  In  performance 
In  the  selected  group.  Further,  the  gain,  in  standard  criterion  units,  that 
will  result  from  selection  can  be  estimated  using  existing  prediction  (i.e., 
decision)  models  if  a  cutting  score  is  set  on  the  predictor.  Brogden  also 
argued  that  a  desirable  metric  for  performance  and  performance  gain  would  be 
to  determine  the  dollar  value  of  variability  in  performance. 

It  remained  for  Cronbach  and  Gleser  (1965)  to  add  the  consideration  of 
selection  costs  and  to  portray  the  utility  of  selection  benefits  in  terms  of 
the  dollar  value  of  performance  increases  minus  the  costs  of  selection. 
Cronbach  and  Gleser  also  elaborated  the  utility  formulation  to  Include  more 
complex  selection  modes  (e.g.,  multiple  hurdles)  and  made  an  attempt  to 
formulate  classification  decisions  in  utility  theory  terms. 


l he  application  of  this  kino  of  uii lity/decision  theory  to  Selection 
and  classification  problems  was  hampered  by  the  difficulty  of  estimating  the 
variability  of  performance  in  dollars,  which  is  a  major  parameter  in  the 
model.  Schmidt,  Hunter,  HcKenzie,  and  Muldrow  (1979)  proposed  a  rather 
simple  solution  in  which  supervisors  are  used  as  judges  to  scale  individual 
performance  in  dollar  terms  via  a  magnitude  estimation  technique.  Judges 
are  asked  to  estimate  the  dollar  payoff  to  the  organization  of  performance 
at  the  50th  percentile  and  the  85th  percentile  for  people  In  the  job  in 

Juestion.  The  difference  between  the  two  estimates  Is  taken  as  the  standard 
eviation  of  individual  performance  in  dollar  terms  (SOy).  So  far,  not  much 
attention  has  been  paid  to  the  basis  on  which  supervisors  make  such 


84 


judgments  although  the  value  for  SOy  1*  frequently  between  40  and  60  percent 
of  the  annual  salary  for  the  position. 

Casclo  (1962b)  has  proposed  another  technique  for  estimating  SOy  In 
dollars  that  also  uses  expert  Judgment  and  1$  tied  explicitly  to  salary. 

Job  analysis  Is  used  to  determine  the  major  task  components  of  t  job.  their 
relative  Importance  Is  determined  by  expert  judgment,  and  a  magnitude 
estimation  technique  Is  used  to  rate  every  person's  performance  on  each  task 
factor.  Average  total  salary  Is  apportioned  to  each  factor  In  accordance 
with  Its  Importance  welaht.  Average  performance  Is  set  equal  to  1.0  and  the 
resulting  scale  Is  multiplied  by  tne  proportion  of  salary  designated  for 
that  factor.  Performance  differences  have  thus  beet,  converted  to  a  dollar 
metric  and  the  standard  deviations  of  the  aggregate  differences  are  put  into 
the  Cronbach  and  Gleser  equation. 

Utility  Judgments  In  the  Military- Context 

Two  principal  factors  make  It  difficult  to  apply  the  previous  work  on 
utility  metrics  and  utility  estimation  to  the  Army  context.  First, 
compensation  practices  are  quite  different  in  the  Army  in  comparison  with 
the  civilian  sector.  Salaries  do  not  differ  by  NOS  and  thus  cannot  be  used 
as  an  index  of  the  job's  relative  worth  to  the  organization.  Second,  while 
industrial  firms  are  In  business  to  provide  products  or  services  so  as  to 
maximize  profit,  the  Army's  overall  mission  is  to  be  prepared  to  defend  the 
United  States  against  military  threats  that  everyone  hopes  will  never  come; 
It  is  difficult  to  try  to  put  a  monetary  value  on  success  or  failure  or  to 
even  think  of  the  utility  of  jobs  in  terms  of  their  monetary  benefit. 

While  dollars  may  not  be  an  appropriate  metric  with  which  to  evaluate  a 
new  Army  classification  system  aimed  at  maximizing  preparedness  for 
catastrophic  events,  resources  are  not  unlimited.  Choices  among  alternative 
personnel  practices  must  be  made,  whether  or  not  there  Is  an  explicit 
utility  metric  on  which  to  make  comparisons. 

The  Air  Force  Procedure.  One  operational  answer  to  the  evaluation 
problem  is  the  system  currently  in  use  in  the  U.S.  Air  Force.  Entry-level 
assignments  in  the  Air  Force  are  made  by  the  PROMIS  selection  and 
classification  system  (Ward,  Haney,  Hendrix,  I  Pina,  1978).  In  brief  terms, 
the  individual  assignment  is  a  function  of  the  following  five  parameters: 

(1)  The  level  of  predicted  training  success,  using  the  ASVAB 
and  other  applicant  Information  as  predictors. 

(2)  The  Individual's  job  preferences. 

(3)  The  rate  at  which  the  targeted  quota  for  a  job  is  currently 
being  filled. 

(4)  The  rate  at  which  the  minority  group  targets  for  each  job 
are  being  filled. 
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(5)  The  seeled  importance  value  of  each  combination  of  Job  holder 
Aptitude  Level  by  Job  Difficulty. 

It  is  this  last  parameter  that  serves  as  the  analog  for  a  utility 
metric  In  the  Air  Force  system.  Previous  scaling  research  using  expert 
judges  has  produced  an  overall  scale  value  for  the  relative  in^ortance  of 
each  combination  of  job  difficulty  (as  determined  by  expert  judgment)  and 
the  aptitude  level  of  a  job  holder  (as  determined  by  ASVAB  scores).  In 
general,  the  Greater  the  job  difficulty  or  the  higher  the  aptitude  level  of 
the  individual,  the  higher  the  value  of  that  personnel  assignment.  However, 
the  prediction  surface  that  relates  the  aptitude  level/difficulty  level 
combination  to  assignment  value  is  not  a  linear  plane. 

The  approach  of  Project  A  to  the  problem  is  similar  but  not  the  same. 
Instead  of  scaling  the  relative  Importance  of  job  difficulty  by  aptitude 
level  combinations,  the  focus  in  Project  A  has  been  on  assessing  the 
differential  value,  or  payoff,  from  HOS-by-predicted-performance-level 
combinations. 

Specific  Utility  Issues  for  Project  A.  The  broad  objective  of  Project 
A  is  to  produce  the  information  necessary  to  develop  a  functional  personnel 
classification  system  for  all  enlisted  personnel.  The  objectives  of  its 
companion  research  endeavor.  Project  B,  are  to  develop  the  necessary 
algorithms  for  relating  labor  supply  forecasts,  applicant  Information,  and 
forecasts  of  system  needs  in  an  assignment  system  that  uses  Project  A  data 
in  an  optimal  fashion.  That  is,  whatever  the  increments  in  selection  and 
classification  validity  produced  by  procedures  developed  in  Project  A,  the 
Project  B  systems  should  allow  investigation  of  how  to  maximize  the  benefit 
from  using  the  new  procedures. 

Within  this  context,  the  utility  problem  for  Project  A  becomes  one  of 
assigning  utility  values  to  MOS  by  Performance  Level  combinations.  That  is, 
If  it  1$  true  that  personnel  assignments  will  differ  in  value  to  the  Army 
depending  on  the  specific  MOS  to  which  an  assignment  is  made  and  on  the 
level  at  which  an  individual  will  perform  in  that  MOS,  then  the  value  of  a 
classification  strategy  that  has  a  validity  significantly  greater  than  zero 
will  increase  to  the  extent  that  the  differential  values  (utilities)  can  be 
estimated  and  made  a  part  of  the  assignment  system. 

Per  Project  A  the  problem  of  estimating  such  utility  values  breaks  down 
Into  a  number  of  specific  questions: 

First,  how  should  performance  levels  be  defined?  Should  it  be  in  terms 
of  some  general  performance  dimension  that  Is  left  unspecified  and  Is 
defined. only  In  terms  of  relative  level  (e.g.,  percentiles)?  Should  a 
general  performance  dimension  be  explicitly  defined,  perhaps  with  behavioral 
anchors  developed  via  critical  Incident  methodology?  Should  individual 
performance  components  be  defined  and  then  explicitly  weighted  for 
combination  into  a  total  score?  All  of  these  are  possibilities  and  a 
specific  research  question  concerns  how  performance  levels  should  be  defined 
and  described  in  the  MOS  by  Performance  Level  combinations. 
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Second,  what  is  the  most  appropriate  Metric  for  describing  the  relative 
value,  or  utility,  of  differential  assignments  across  MOS/Performance  Level 
combinations?  Previous  work  in  personnel  psychology  has  been  linked  almost 
exclusively  to  a  dollar  Metric  and  has  tried  to  estimate  the  variability  in 
payoff  froM  people  at  different  performance  levels  in  dollar  terms,  but  only 
in  a  selection  context.  Estimating  differential  payoff  from  a  systemwide 
classification  system  remains  unexplored.  Since  the  dollar  Metric  appears 
to  be  inappropriate  for  the  Army  context  and  because  there  is  little 
previous  work  on  applying  utility  theory  to  personnel  classification,  the 
Metric  question  for  Project  A  is  a  very  difficult  one.  It  suggested  an 
exploratory  approach. 

Third,  assuming  the  question  of  the  metric  is  resolved,  the  specific 
method(s)  to  be  used  for  estimating  differential  assignment  utility  in  the 
appropriate  metric  must  then  be  considered.  Only  two  options  see»  even 
possible.  In  the  first,  it  might  be  possible  to  relate  the  performance  of. 
individuals  or  units  to  some  kind  of  ^bottom  line*  measure  that  Army 
managenent  would  consider  an  appropriate  metric.  For  example,  realistic 
field  exercises  could  be  used  to  determine  the  relationships  of  Individual 
performance  measures  to  the  performance  of  a  unit  in  a  simulated  engagement. 
The  difficulties  with  this  approach  revolve  around  the  expense  of  collecting 
such  data,  the  necessity  of  having  such  exercises  for  each  NOS,  and  the  need 
to  equate  scores  in  some  way  across  NOS. 

A  second  alternative  is  to  turn  to  scaling  technology  and  use  expert 
judges  to  estimate  the  relative  value  of  differential  personnel  assignments. 
Since  a  variety  of  scaling  models  and  scaling  techniques  are  available,  a 
major  problem  would  be  to  choose  the  procedure  that  is  feasible,  makes  the 
best  use  of  the  information  held  by  the  judges,  and  provides  sufficient 
internal  validity  information  to  generate  confidence  and  acceptability  for 
the  scale  values. 

Because  the  above  questions  are  difficult  ones  and  have  been  largely 
unresearched  in  the  past,  the  plan  that  was  developed  for  addressing  them 
was  exploratory  in  nature.  Its  goal  was  to  proceed  from  a  very  broad 
consideration  of  a  number  of  methods  to  a  focus  on  a  procedure  that  Is 
valid,  feasible,  and  acceptable  to  the  Army. 


GENERAL  APPROACH 

Phase  one  consisted  of  a  series  of  seven  small  group  workshops  with 
Armw  officers  (Sadacca  &  Campbell,  1987).  The  workshops  were  designed  to 
explore  a  number  of  issues  pertaining  to  utility  metrics,  utility 
estimation,  and  the  definition  of  performance  levels.  Each  workshop  was 
divided  into  a  period  for  trying  out  prototypic  judgment  tasks  and  a  period 
for  open-ended  discussion  of  issues. 

Although  the  atmosphere  was  informal  and  the  participants  were  free  to 
bring  up  any  questions  or  Issues  they  wished,  six  questions  were  used  to 
guide  the  discussions. 
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(1)  How  shall  Measures  of  performance  be  weighted  and  overall 
performance  defined? 

(2)  What  kinds  of  scaling  judgments  can  officers  reasonably 
be  asked  to  make? 

(3)  Are  there  major  scenario  effects  on  performance  factor 
weights  and  utility  judgments? 

(4)  In  what  metric  should  the  utility  of  enlisted  personnel 
assignments  be  expressed? 

(5)  What  is  the  form  of  the  relationship  between  performance 
and  utility  within  MOS? 

(6)  Who  will  sake  the  best  judges  for  the  final  scaling? 

The  prototypic  judgment  tasks  that  were  tried  out  in  this  first  phase 
were  of  the  following  general  nature: 

(1)  Assignment  of  importance  weights  to  performance  factors. 

(2)  Rank  ordering  of  overall  utility  of  MOS  by  Performance 
Level  combinations  when  performance  was  defined  in 
percentile  terms. 

(3)  P.atio  judgments  of  comparative  utility  for  different  MOS 
by  Performance  Level  combinations. 

The  specific  reactions  of  each  participant  to  the  sample  scaling  tasks  were 
also  used  as  items  for  general  discussion. 

The  second  phase  of  the  research  was  devoted  to  solving  the  practical 
problems  of  assigning  utility  values  to  performance  levels  in  the  broad 
array  of  entry-level  MOS.  Additional  workshops  were  conducted  to  try  out 
various  scaling  methods  and  to  prepare  for  the  third  phase,  in  which  the 
selected  scaling  methods  were  applied  to  entry-level  MOS  and  within  MOS 
performance  levels. 


THE  EXPLORATORY  WORKSHOPS 

In  this  first  phase  there  was  no  vigorous  testing  of  hypotheses,  no 
experimental  design  or  testing  for  statistical  significance.  If  something 
didn't  seem  to  work  it  was  dropped  or  modified;  if  something  else  was 
suggested  it  was  tried  out.  The  overall  intent  was  to  determine  what  was 
possible,  before  being  concerned  with  how  to  do  it  most  effectively. 

MorhshgpJ 

A  critical  initial  concern  was  whether  Army  officers  would  be  willing 
to  make  evaluative  judgments  comparing  the  utility  of  enlisted  soldiers  in 
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different  MOS.  Officers  light,  for  example,  argue  that  all  illltary  Jobs 
are  essential,  and  that  It  does  not  lake  sense  to  say  that  the  soldier  who 
transports  the  ammunition  has  any  less  utility  than  the  soldier  wno  fires 
the  weapon,  or  the  soldier  who  treats  the  wounded,  or  the  soldier  who 
prepares  seals. 

Another  concern  was  what  illltary  situation  or  scenario  shrt'ld  be  used 
as  the  context  In  which  utility  was  to  be  Judged.  It  seeied  very  reasonable 
to  believe  that  the  utility  to  the  Any  of  different  Military  Jobs,  and 
performance  levels  within  those  Jobs,  would  vary  as  a  function  of  the 
stipulated  Military  situation. 

A  third  concern  centered  on  what  considerations  enter  Into  utility 
Judgments  Made  by  Any  officers.  When  evaluating  a  soldier's  utility,  what 
contributions  to  Mission  accomplishment  do  officers  emphasize? 

To  get  an  Initial  understanding  of  the  various  issues,  it  was  decided 
not  to  spell  out  any  Military  context  for  Making  the  utility  Judgments  to 
officers  attending  the  first  workshop  (six  field  grade  officers  fro*  the 
Any  Research  Institute).  The  purpose  was  to  find  out  whether  these 
officers  would  evoke  their  own  Military  context  for  the  judgments,  and  If 
so,  what  context  they  would  choose.  To  assess  the  reasonableness  of  Baking 
anv  utility- type  judgments,  the  only  scaling  task  used  was  simply  to  ask  for 
a  rank  ordering  of  MQS/perf omance  level  co«b1nations,  rather  than  for  More 
sophisticated  judgments  that  could  yield  an  Interval  or  ratio  utility  scale. 

After  a  brief  Introduction  to  Project  A  and  a  discussion  of  the  concept 
of  Job  performance  utility,  the  six  officers  were  c  /en  the  task  of  rank 
ordering  a  set  of  57  enlisted  MOS/performance  level  conin’ nations  froM  the 
set  of  19  Project  A  MOS  Involved  in  the  Concurrent  Validation  testing  (see 
"Initial"  portion  of  Figure  4.1).  To  facilitate  their  judgments,  they  were 
provided  with  a  separate  listing  of  summary  job  descriptions  for  each  MOS. 

Perhaps  the  most  Important  result  was  si*ply  that  the  officers  were 
willing  to  do  the  task.  They  did  not  argue  that  it  was  an  unreasonable  one 
and  seemed  to  undertake  the  task  quite  seriously  and  carefully. 

Another  significant  result  emerged  In  the  post-task  discussion: 
Independently,  each  of  the  six  officers  had  chosen  the  same  setting  —  that 
of  a  conflict  In  Europe  —  as  the  context  in  which  he  had  rank  ordered  the 
utility  of  the  MOS/performance  level  combinations.  The  officers  expressed 
the  opinion  that  the  Army's  principal  current  mission  is  to  ready  itself  for 
such  a  possibility.  They  also  agreed  that  had  we  spelled  out  a  peacetime  or 
a  different  wartime  context,  their  utility  rankings  would  most  likely  have 
been  different.  However,  In  their  opinion,  even  If  we  used  a  peacetime 
scenario  it  should  be  one  that  emphasized  training  and  other  readiness 
activities  geared  toward  the  outbreak  of  hostilities  In  Europe. 

The  rank  order  Intercorrelations  aamng  the  officers  were  computed 
across  the  57  MOS/performance  level  combinations.  These  correlations  ranged 
fro®  .29  to  .90  with  an  average  of  .69.  These  results  were  heartening, 
since  they  Indicated  that  quite  reliable  (.95  or  above)  average  utility 
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Figure  4.1.  Description  of  soldier  performance. 
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ranks  could  be  obtained  by  using  10  or  more  Judges.  The  results  also 
Indicated  that  there  Bay  be  a  fairly  common  frame  of  reference  aaong  Army 
officers  in  their  evaluation  of  MOS/perforaance  level  utilities. 

In  the  aanner  of  a  subjective  expected  utility  aodel,  the  officers  were 
next  asked  to  evaluate  the  relative  priority  of  eight  outcomes  of  a  ailitary 
engageaent  that  could  result  froa  effective  performance  of  enlisted 
personnel  In  that  situation.  The  eight  outcoaes  (increased  force  survival, 
enhanced  readiness,  enhanced  efficiency  or  cost-effectiveness,  enhanced 
aobility  and  firepower,  enhanced  physical  and  psychological  well-being. 
Increased  local  civilian  cooperation  and  support,  decreased  capability  and 
performance  of  enemy  units,  enhanced  performance  of  supporting  Army  units) 
were  chosen  by  the  research  staff  without  regard  to  any  official  Army 
doctrine.  In  this  section  of  the  workshop  a  ailitary  scenario  (chosen 
before  the  workshop  began)  was  specified— a  scenario  describing  the 
outbreak  of  hostilities  in  Europe  that  had  been  used  previously. in  Project  A 
activities  (see  Figure  4.2). 

In  the  first  variant,  the  officers  rank  ordered  the  eight  outcoaes; 
then,  assigning  ten  points  to  the  lowest  ranked  outcome,  they  assigned 
points  to  the  reaalnlng  outcoaes  In  accordance  with  the  perceived  ratio  of 
their  laportance  to  the  lowest  ranked  outcoae.  For  a  second  aethod,  the 
officers  were  presented  the  eight  outcoaes  In  a  paired-comparison  format; 
for  each  possible  pair  of  outcoaes,  their  task  was  to  divide  10Q  points 
between  the  outcoaes  in  a  Banner  that  reflected  the  outcomes'  relative 
laportance  in  the  given  ailitary  situation. 

The  officers  distinctly  did  not  like  the  paired-comparison  format, 
feeling  that  It  was  like  a  test  of  their  consistency  In  assigning  laportance 
points. 

In  the  discussion  period,  the  officers  declared  that  dollar  cost 
considerations  had  no  piece  on  a  battlefield,  that  losing  or  even  winning  a 
war  could  not  be  evaluated  in  dollar  terms.  They  further  indicated  that  the 
costs  of  training  and  equipping  soldiers  did  not  enter  into  their  rankings 
of  HOS/perfcrmance  level  utility. 

In  response  to  the  question  of  whether  judges  should  evaluate  MQS/ 
performance  levels  against  separate  utility  dimensions,  the  officers 
expressed  a  clear  preference  for  making  one  overall  utility  rating.  They 
also  felt  that  the  description  of  the  HOS/performance  levels  should  be  kept 
general  rather  than  aade  more  specific. 


The  second  and  third  workshops  were  scheduled  back-to-back  on  suc¬ 
cessive  days,  with  the  intent  of  using  the  same  stimulus  materials  and 
judgment  tasks  In  both  workshops.  However,  discussions  with  the  officers  in 
the  second  workshop  led  to  changes  in  the  procedures  used  the  next  day. 


WARTIME  SCENARIO:  FIRST  AMO  SECOND  WORKSHOPS 


Your  unit  Is  assigned  to  a  U.S.  Corps  in  Europe.  Hostilities  have  broken 
out  and  the  Corps  coabat  units  are  engaged.  The  Corps  'Mission  is  to 
defend,  then  re-establish,  the  host  country's  border.  Pockets  of  eneay 
airborne/helicopter  and  guerilla  eleaants  are  operating  throughout  the  Corps 
sector  area.  The  Corps  Maneuver  terrain  is  rugged,  hilly,  and  wooded,  and 
weather  Is  expected  to  be  wet  and  cold.  Halted  initial  and  reactive 
cneaical  strikes  have  been  eaployed  but  nuclear  strikes  have  not  been 
initiated.  Air  parity  does  exist. 


WARTIME  SCENARIO:  THIRD  -  SEVENTH  WORKSHOPS 

Hostilities  have  broken  out  in  Europe  and  your  Corps'  coabat  units  are 
engaged.  Your  Corps'  Mission  is  to  defend,  then  re-establish,  the  host 
country's  border.  Pockets  of  eneay  airborne/heliborne  and  guerilla  eleaents 
are  operating  throughout  the  Corps  sector  area.  Halted  Initial  end 
reactive  cheaicals  strikes  have  been  eaployed  but  nuclear  strikes  have  not 
been  initiated.  Air  parity  does  exist. 


PEACETIME  SCENARIO:  FOURTH  -  SEVENTH  WORKSHOPS 

Europe  is  In  the  peacetime  condition  currently  prevailing  there.  Your 
Corps'  aission  is  to  defend  and  maintain  the  host  country's  border  should 
war  break  out.  The  potential  eneay  approxlaates  a  combined  arms  a nay  and 
has  nuclear  and  cheoical  capability.  Air  parity  does  exist.  The  Corps  has 
personnel  and  equipaent  sufficient  to  sake  its  aission  capable  for  training 
and  evaluation.  The  training  cycle  Includes  periodic  field  exercises, 
command  and  Maintenance  inspections,  ARTEP  evaluations,  and  Individual 
soldier  training/SQT  testing. 


Figure  4.2.  Scenarios  used  in  exploratory  utility  workshops. 
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One  such  change  involved  the  scenario  used  to  describe  the  military 
context  for  the  utility  judgments.  Discussions  with  the  six  field  grade 
officers  in  the  second  workshop  indicated  that  their  utility  ratings  sight 
well  have  been  influenced  by  the  type  of  unit  to  which  they  imagined 
themselves  assigned.  Furthermore,  they  might  have  been  responding 
differentially  to  the  “rugged,  hilly  and  wooded*  terrain  description.  One 
officer,  for  example,  reported  that  he  had  downgraded  the  utility  of  armor 
crewmen  because  of  the  more  limited  use  of  tanks  in  that  setting,  while 
other  officers  reported  th/t  they  had  nevertheless  assigned  very  high 
utility  values  to  the  Armor  Crewman  MOS. 

The  officers  suggested  keeping  the  scenario(s)  free  of  specific  details 
that  would  favor  one  MOS  at  the  expense  of  another.  The  references  in  the 
scenario  to  the  specific  terrain  and  weather  conditions  were  therefore 
deleted  from  the  wartime  scenario  used  in  the  third  and  subsequent 
workshops.  Moreover,  the  military  unit  of  concern  was  made  the  entire 
Corps,  rather  than  an  unspecified  unit  within  the  Corps.  (See  Figure  4.2.) 

In  both  the  second  and  third  workshops,  verbal  descriptions  of 
MOS/performance  level  combinations  were  used.  The  descriptions  were  the 
same  as  those  used  In  the  first  workshop,  with  one  exception:  The  overall 
performance  scale  was  changed  from  one  that  was  behavioral ly  anchored  to  one 
expressed  in  percentiles  (see  Figure  4.1).  This  change  was  made  In 
recognition  of  the  difficulty  of  assigning  performance-based  anchors  that 
would  be  comparable  across  MOS  In  the  absence  of  actual  performance  data. 

In  the  second  and  third  workshops,  in  addition  to  rank  ordering  the 
described  soldiers,  the  participating  officers  were  asked  to  assess  the 
relative  utility  of  each  o?  the  soldiers  in  comparison  to  one  particular  «> 
standard  soldier  whose  utility  was  arbitrarily  set  at  100.  The  officers 
compared  each  of  the  56  remaining  soldiers  In  turn  to  the  standard  soldier 
and  assigned  a  proportionate  utility  value  to  each,  given  that  the  standard 
soldier's  value  was  set  at  100.  Two  standard  soldiers  were  used:  the  90th 
percentile  Infantryman  (MOS  11B)  and  the  50th  percentile  Annunition 
Specialist  (MOS  55B ) .  These  two  MOS/performance  level  combinations  were, 
respectively,  rank  ordered  very  high  and  near  the  median  by  the  first 
workshop  officers.  The  officers  were  allowed  to  assign  zero  utility  values 
or  even  negative  values  if  they  thought  the  soldier  described  would  detract 
from  mission  accomplishment. 

The  average  inter judge  correlations  and  correlations  between  like 
utility  measures  across  workshops  were  sufficiently  high  to  suggest  that 
very  reliable  average  rank  and/or  ratio  scale  values  could  be  obtained  using 
about  10  judges.  The  high  Intercorrelations  among  the  different  measures 
suggested  that  the  final  utility  scaie  values  (with  appropriate 
transformations)  might  be  fairly  similar  across  measurement  methods. 

It  was  also  apparent  that,  on  the  average,  the  combat  MOS  received 
higher  utilities  than  the  noncombat  MOS  at  all  three  performance 
percentiles. 


Differences  In  average  scale  values  between  the  90th  and  50th 
percentile  soldiers  and  between  the  50th  and  10th  percentile  soldiers 
suggested  some  Important  nonlinear  relationships  between  performance  and 
utility,  which  many  Investigators  assume  to  be  linear.  Discussions  with  the 
officers  in  the  workshops  supported  the  nonlinear  view. 

In  the  discussion  following  the  Judgment  tasks,  the  participants  showed 
clear  preference  for  the  90th  percentile  Infantryman  rather  than  the  50th 
percentile  Ammunition  Specialist  as  an  anchor,  in  part  because  they 
considered  It  easier  to  scale  other  HOS  between  the  0  and  100  points,  and  In 
part  because  Infantryman  is  the  most  common  and  best  known  Army  M0S. 

When  asked  whet  major  factors  they  considered  In  assigning  utilities  to 
the  MOS/perf ormonce  combinations  for  the  wartime  scenario  given,  the 
officers  Indicated  that  potential  contribution  to  unit  survival  and 
.usefulness  In  replacing  troop  losses  were  foremost.  This  was  consistent 
with  the  ratings  given  by  the  Workshop  1  officers  of  the  relative  Importance 
of  various  outcomes. 


When  asked  how  general  or  specific  the  descriptions  of  the 
MOS/performance  levels  should  be,  the  workshop  participants  said  that  most 
officers  think  In  terms  of  top,  bottom,  and  mid-level  enlisted  personnel. 
That  Is,  a  soldier  Is  either  good,  poor,  or  somewhere  In  the  middle.  They 
felt  that  very  general  performance  descriptions  would  best  capture  this 
outlook. 


Workshops  4  and  5 

The  fourth  and  fifth  wurk&hop*  were  conducted  for  the  must  part  with 

the  field  grade  officers  who  had  participated  In  the  first  and  third 
workshops.  The  officers  at  both  workshops  were  asked  to  follow  new 
procedures  that  had  not  been  tried  out.  Using  the  same  wartime  scenario  and 
the  57  MQS /performance  level  combinations  used  In  the  third  workshop,  the 
officers  were  asked  to  judge  228  pairs  of  MOS/performance  level 
combi nations. 


The  judgments  were  of  the  form: 

( _ )  soldiers  of  MOS/performance  level  combination  1  are  equal 

In  overall  value  to  the  Corps  in  the  wartime  military  situation  as 
( _ )  soldiers  of  performance  level  combination  2, 

The  judgmental  task  was  to  fill  In  the  two  blanks  with  numbers  that  would 
make  the  two  types  of  soldiers  equal  in  value.  For  example,  If  the  two 
MOS/performance  level  combinations  were  90th  percentile  Utility  Helicopter 
Repairer  (M0S  67N)  and  50th  percentile  Combat  Engineer  (M0S  12B),  an  officer 
might  judge  that  seven  of  one  type  would  be  wortn  five  of  the  other.  The 
**f fleers  were  allowed  to  put  in  any  number  they  liked  In  order  to  make  the 
vo  groups  of  soldiers  equal  in  worth. 
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The  228  pairs  of  MOS/performance  level  combinations  consisted  of  two 
types:  (a)  57  pairs  In  which  each  pair  ■sober  was  fro*  the  same  MOS  but  at 
a  different  performance  level,  that  Is,  10th,  50th,  or  90th  percentile  (19 
MOS  x  3  pairs  —  10-50,  10-90,  and  50-90);  and  (b)  171  pairs  in  which  each 
pair  member  was  from  a  different  one  of  the  19  W)S,  with  one  performance 
level  for  each  MOS  (19  x  18/2  •  171).  The  228  pairs  were  randomized  and 
then  presented  In  the  same  order  to  all  judges. 

Scale  values  for  each  of  the  MOS/performance  level  combinations  making 
up  the  171  judgmental  pairs  were  calculated  using  a  ratio  scaling  procedure 
described  by  Torgerson  (1958,  pp.  105-112).  This  procedure  results  In  a  set 
of  scale  values  whose  geometric  mean  Is  equal  to  1.0. 

Table  4.1  presents  the  average  of  the  officers'  scale  values  obtained 
for  the  57  MOS/performance  level  combinations  using  the  pal red- comparison 
ratio  scaling  technique  described  above.  Consistent  with  earlier  findings, 
the  combat  MOS  generally  have  higher  utility  ratings  at  each  of  the  three 
performance  levels  (10tn,  50th,  and  90th  percentile)  than  the  noncombat  MOS. 
However,  the  difference  in  utility  scale  values  within  an  MOS  from  the  90th 
to  50th  percentile  performance  level  Is  greater  for  all  19  MOS  than  the 
difference  in  utility  scale  values  from  the  50th  to  10th  percentile 
performance  level.  This  is  especially  evident  for  the  combat  MOS  which,  on 
the  average,  showed  the  greatest  declines  in  utility  values  from  the  90th  to 
50th  percentile  performance  levels. 


The  inconsistency  of  these  .results  with  those  cited  earlier  may  be 
attributable  more  to  the  scaling  method  used  than  tc  the  sample  of  officers 

involved,  since  the  officers  whose  judgments  were  pooled  to  arrive  61 
Workshop  5  scale  values  overlapped  considerably  with  the  officers  In 
Workshop  3. 


al. 


The  average  Interjudge  correlation  between  the  scale  values  of  the 
eight  officers  taken  across  the  57  combinations  was  .61.  This  value,  though 
not  as  high  as  that  obtained  for  the  scaling  methods  tried  out  in  Workshop 
3,  was  considered  encouraging  enough  to  try  out  the  scaling  method  again  in 
Workshops  6  and  7. 


As  five  of  the  six  officers  In  Workshop  5  had  rank  ordered  the  57  MOS/ 
performance  level  combinations  usino  the  same  wartime  scenario  as  in  Work¬ 
shop  3,  one  and  one-half  months  earlier,  it  was  of  Interest  to  determine  how 
reliable  their  average  rankings  were.  The  correlation  between  the  first  and 
second  average  rankings  by  the  five  officers  across  the  57  combinations  was 
.98. 


Another  indication  of  the  stability  of  the  average  rankings  Is  the 
average  Interjudge  correlation  obtained  among  the  rank  orders  of  the  six 
officers.  The  obtained  average,  .79,  Is  slightly  higher  than  the  average 
obtained  for  Workshop  3  (.75).  Both  average  Inter judge  correlations 
Indicate  that  the  average  rank  ordering  based  on  10  judges  would  probably 
have  a  reliability  of  .95  or  better. 
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Table  4.1 


Seal#  Valuta  of  NOS/Ptrforaanct  Itvtl  Hypothetical  Soldltr* 
(50th  Percentile  Infantryman  -  1.0;  ■  «  8,  Workshops  4  and  5) 


Percentile  _  foflkJPiliar.&nfiS 


El 

USL 

-SSL 

-SSL 

(90-50) 

Administrative  Specialist  (711) 

.10 

.23 

.46 

.23 

.13 

Ammunition  Specialist  (558) 

.17 

.49 

1.01 

.52 

.32 

Carpentry/Masonry  Specialist  (518) 

.09 

.21 

.43 

.22 

.12 

Chemical  Operations  Specialist  (54E) 

.26 

.70 

1.51 

.81 

.44 

Food  Service  Specialist  (948) 

.10 

.23 

.53 

.20 

.13 

light  Wheel  Vehicle  Mechanic  (636) 

.16 

.43 

.75 

.32 

.27 

Medical  Specialist  (918) 

.21 

.58 

1.29 

.71 

.37 

Military  Police  (958) 

.17 

.34 

.66 

.32 

.17 

Motor  Transport  Operator  (64C) 

.12 

.37 

.68 

.31 

.25 

Petrol.  Supply  Specialist  (76W) 

.13 

.31 

.71 

.40 

.18 

Single  Channel  Radio  Operator  (Q5C) 

.15 

.41 

.91 

.50 

.26 

TOW/Dragon  Repairer  (27E) 

.23 

.64 

1.26 

.62 

.41 

Unit  Supply  Specialist  (76Y) 

.08 

.23 

.45 

.22 

.15 

Util.  Heli.  Repairer  (67N) 

.17 

.52 

1.06 

.54 

.35 

Noncombat  M0$  Average 

.42 

.25 

Armor  Crevanan  (19E/K) 

.42 

1.28 

2.71 

1.43 

.86 

Cannon  Crewman  (13B) 

.29 

.75 

1.53 

.78 

.46 

Combat  Engineer  (12B) 

.26 

.72 

1.46 

.74 

.46 

Infantryman  (UB) 

.34 

1.00 

2.01 

1.01 

.66 

MANPADS  Crewman  (16S) 

.27 

.72 

1.26 

.54 

.45 

Combat  M0S  Average 

.90 

.58 

After  the  tlx  officers  in  Workshop  5  finished  scaling  the  MOS/perfor- 
aancc  level  combinations,  they  were  asked  to  re-rank  the  57  combination 
cards  under  a  peacetime  scenario  (see  Figure  4.2).  The  peacetime  scenario 
Mas  set  in  Europe  under  current  conditions  and  eaphasized  maintaining  force 
readiness.  Table  4.2  shows  the  NOS/perfomance  level  combinations  having 
differences  in  average  assigned  rank  of  10  or  sore  under  the  wartime  versus 
peacetime  scenarios.  The  trend  In  the  data  from  the  six  officers  is  clear: 
Low-perforaance-level  coabat  troops  art  ranked  higher  In  wartiac  than 
peacetlae,  while  high-performance-level  support  personnel  are  ranked  lower 
In  wartime  than  peacetiac. 

The  differences  in  average  utility  ranks  found  in  Table  4.2  Are 
certainly  not  surprising.  They  raise  the  question  of  how  a  computerized 
selection  and  assignaent  procedure  can  best  use  utilities  If  such  utilities 
are  in  soae  part  a  function  of  the  context  In  which  the  judgaents  of  utility 
are  aade.  It  aay  be  necessary  to  use  utilities  obtained  through  a  nuaber  of 
scenarios  or  to  decide  upon  one  particular  scenario  as  thr  context  for  the 
utility  judgaents.  On  the  other  hand,  if  the  differences  are  not  large, 
there  aay  not  be  a  significant  difference  in  the  recommended  assignments  to 
Aray  jobs  using  utilities  obtained  under  different  scenarios.  The  correla¬ 
tion  across  the  57  combinations  of  the  average  rank  assigned  by  the  six 
officers  under  the  wartiae  and  peacetlae  scenarios  was  .85.  Computer 
simulations  using  different  utility  values  and  realistic  operational 
constraints  aay  eventually  be  needed  to  determine  the  practical  significance 
of  scenario  differences. 

After  the  officers  had  completed  the  judgmental  tasks,  a  nuaber  of 
utility  issues  were  discussed.  The  officers  reported  being  concerned,  when 
using  the  paired-coaparison  ratio  scaling  method,  that  they  were  being 
inconsistent  in  assigning  numbers  across  the  judgmental  pairs  of  MOS/perfor- 
aance  level  coobinations.  They  were  assured  that  inconsistency  could  be 
expected  within  that  type  of  judgment  series.  (The  instructions  were  later 
modified  in  Workshops  6  and  7  to  stress  that  it  was  not  necessary  to  strive 
for  consistency  in  aaklng  these  kinds  of  judgaents.) 

When  asked  what  WOS/perforaance  level  soldiers  might  best  be  used  as  a 
standard  or  unit  in  measuring  the  utility  of  other  soldiers,  the  officers 
agreed  that  the  50th  percentile  Infantryman  would  be  the  best  choice.  They 
felt  that  not  only  are  there  more  Infantrymen  than  soldiers  in  any  other 
MOS,  but  that  officers  in  general  have  a  good  understanding  of  what  an 
average  Infantryman  is  like  and  what  he  can  do. 

The  officers  were  also  asked  what  their  reaction  would  be  to  expressing 
the  differential  worth  or  utility  of  soldiers  in  terns  of  dollars.  They 
reacted  very  negatively  to  this  concept,  citing  possible  adverse  political 
consequences  as  well  as  internal  Army  morale  problems  if  dollar  figures  were 
placed  on  soldiers'  worth. 
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Table  4.2 

HOS/hr formanca  Level  Hypothetical  Soldiers  With  Large  Kean  Wartime  vs. 
Peacetime  Differences  in  Hank  Order  (n  •  I,  Workshop  5) 


HOS/Ptrforwanca  icyftl 

m 

SiHim 

ItortiM  Hiahtr  IhAn-EHgtlin 

Cannon  Crewman,  10th  percentile 

138 

29 

39 

Cannon  Crewman,  50th  percentile 

10 

20 

Chemical  Opers  Spec,  10th  percentile 

54E 

35 

48 

Infantryman,  10th  percentile 

118 

25 

40 

Infantryman,  50th  percentile 

10 

20 

Armor  Crewman,  10tn  percentile 

19E/K 

25 

37 

MANPADS  Crewman,  10th  percentile 

16S 

31 

42 

PeactttiM  Miaher  Than  Wartime 

Administrative  Spec,  10th  percentile 

71L 

56 

45 

Administrative  Spec,  50th  percentile 

46 

28 

Administrative  Spec,  90th  percentile  - 

36 

17 

WBI  pv.il  wi  J  1  nu  g»  wi « •  J  9  wv  vm 

51B 

An 

39 

Carpentry /Masonry  Spec,  90th  percentile 

41 

26 

Food  Service  Spec,  50th  percentile 

94B 

41 

25 

Food  Service  Spec,  90th  percentile 

30 

12 

Unit  Supply  Spec,  90th  percentile 

76Y 

28 

14 

Workshops  6  and  7 

When  the  officers  in  Workshops  6  and  7,  which  were  held  in  Europe,  were 
asked  the  same  question  concerning  the  use  of  a  utility  dollar  metric  their 
reaction  was.  If  anything,  even  more  strongly  negative.  They,  like  the 
officers  in  earlier  workshops,  agreed  that  the  50th  percentile  Infantryman 
would  make  the  best  standard  against  which  the  utility  of  soldiers  in  other 
MOS/perfomance  level  combinations  could  be  judged. 

Thirteen  officers  attended  Workshops  6  and  7.  All  were  captains  and 
majors,  while  the  earlier  workshop  participants  all  had  been  majors  and 
lieutenant  colonels.  The  consistency  of  the  opinions  expressed  by  the 
officers  in  the  discussion  periods,  despite  the  differences  in  grade  levels 
and  locations,  suggests  that  Army  officers  have  a  fairly  well-shared  frame 
of  reference. 
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This  common  viewpoint  mas  also  reflected  In  the  results  of  the  analyses 
of  the  workshop  data.'  The  Workshops  6  and  7  participants  were  asked  to  le&ke 
essentially  the  same  types  of  judgments  Bade  by  earlier  workshop  partic¬ 
ipants.  However,  this  tlae  they  judged  the  utility  of  95  HOS/performance 
level  combinations  (5  performance  levels— 10th,  30th,  50th,  70th,  and  90th 
percentile— for  each  of  the  19  NOS)  Instead  of  57  combinations.  The 
correlation  was  .94  across  the  average  pal red- comparison  ratio  scale  values 
of  the  57  coablnations  that  were  coamon  between  Workshops  4  and  5  (majors 
and  lieutenant  colonels)  and  Workshops  6  and  7  (captains  and  majors). 

The  aeans  of  the  rank  orders  assigned  the  95  HOS/perforaance  level  com¬ 
binations  under  the  war  and  peacetime  scenarios  by  the  13  officers  are  shown 
in  Table  4.3.  The  H0S  in  the  table  have  been  placed  In  three  groups  based 
on  comparative  rankings.  The  first  group  contains  mostly  combat  NOS.  All 
the  NOS/perforaance  level  combinations  Involving  these  HOS  had  higher  aver¬ 
age  rank  orders  under  the  wartime  than  under  the  peacetime  scenario.  In  the 
second  group  of  NOS  all  the  WS/performance  level  combinations  were  ranked 
higher  under  the  peacetime  than  the  wartime  scenario.  In  the  third  group  of 
NOS  the  average  rank  orders  of  the  HOS/performance  level  combinations  were 
all  higher  under  peacetime  than  wartime  at  the  upper  levels  of  performance, 
but  were  all  lower  under  peacetime  than  wartime  at  the  lower  levels  of 
performance.  Soldiers  In  these  NOS  generally  have  a  higher  probability  of 
being  in  a  combat  situation  than  soldiers  In  the  second  group  of  NOS. 

These  data  were  consistent  with  the  Workshop  4  and  5  findings  and  the 
statements  made  during  the  discussion  periods:  Soldiers  at  low  performance 
levels  who  are  likely  to  be  Involved  in  combat  are  assigned  relatively 
higher  utility  under  a  wartime  scenario,  while  soldiers  at  high  performance 
levels  who  are  unlikely  to  be  involved  in  combat  are  assigned  relatively 
higher  utility  under  a  peacetime  scenario.  However,  since  the  correlation 
across  the  95  combinations  of  the  utility  values  under  the  two  scenarios  may 
be  quite  high  (the  correlation  of  average  rank  orders  was  .83  In  the 
Workshops  6  and  7  data  and  .85  for  the  comparable  Workshops  4  and  5  data), 
the  simulations  may  well  result  in  relatively  minor  scenario-derived 
differences. 


In  Workshops  6  and  7,  12  of  the  officers  scaled  the  95  combinations  in 
WO  ways,  vne  method  was  the  pa  1 re u- Cusp a r iSum  ratio  pFOCcuiifc  iiScu  by  the 
Workshops  4  and  5  participants.  They  also  scaled  the  95  combinations  using 
the  subjective  estimation  procedure  employed  by  the  Workshops  3  and  4 
participants.  In  this  method  one  combination  is  given  a  utility  value  of 
100  and  the  other  combinations  are  assigned  scale  values  that  reflect  their 
respective  proportionate  utilities;  the  combination  assigned  the  value  of 
100  was  the  90th  percentile  Infantryman.  The  scales  obtained  by  the  two 
methods  were  then  transformed  to  scales  In  which  the  50th  percentile 
Infantryman  had  a  utility  value  of  1.0. 


Table  4.4  shows  the  scale  values  of  the  95  HOS/performance  level 
combinations  obtained  through  using  both  methods.  The  utility  scale  values 
obtained  from  the  two  methods  are  quite  similar  at  the  lower  performance 
levels.  However,  with  the  exception  of  the  Infantryman  and  Armor  Crewman 
HOS,  the  scale  values  for  the  higher  performance  levels  obtained  from  the 
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Tab1«  4.3 

NMn  Rank  Order  of  KOS/Performnce  Laval  Cc^jI nation*  Under  Wartiaa  and 
Mecetiae  Scenarios  (n  •  13,  Workshop*  6  and  7) 


■fgfgnmsi  Estonia 


w& 

JO- 

JL, 

JOL 

twfmtryma 
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0 

47 

13 

S3 

6 

17 
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8 

IS 

7 

ex 

47 

fi 

16 

Cmm  CmM 

0 

17 

» 
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HI 

21 

8 

16 

OMaical  Operations 

40 

a 

14 

Specialist  (S4C) 

EX 

22 

38 

U 

1  Radio  Opo rotor 

77 

a* 

41 

C 

8 

38 

14 

a 

CoePat  tagleeer 

M 

72 

32 

a 

14 

(121) 

p 

17 

m 

48 

34 

a 

NMPAOS  Crowwi 

V 

74 

S3 

37 

8 

is 

(1SS) 

p 

86 

68 

84 

38 

is 

BoiOad  Bldiar  la  Peacetime  Sceearle 

*fc1  el  strati**  Specialist 

M 

a* 

77 

68 

27 

»•> 

41 

[TIL)  • 

r 

si 

*/ 

40 

M 

7 

Oilt  Supply  Specialist 

M 

12 

73 

24 

42 

0 

(76T) 

P 

76 

81 

39 

C 

7 

light  Wheel  safe  Noch 

W 

71 

<3 

48 

33 

a 

(63*) 

P 

71 

81 

42 

0 

10 

Food  Sorrlco  Specialist 

w 

13 

70 

56 

46 

a 

(»4i) 

p 

ai 

80 

44 

a 

12 

Carpentry/Kesonry  Specialist 

w 

«i 

•  84 

75 

0 

a 

(SIS) 

p 

84 

85 

SO 

34 

a 

iuai 

Medical  Specialist 

w 

74 

S7 

45 

a 

is 

(•11) 

p 

12 

61 

42 

a 

7 

TOK/DreoM  Repairer 
(02) 

M 

P 

80 

83 

82 

•7 

SO 

48 

a 

a 

0 

IS 

Utility  Helicopter  Repairer 
(««) 

M 

78 

61 

45 

a 

a 

P 

11 

85 

43 

a 

a 

Motor  Transport  Operator 

U 

80 

63 

S2 

a 

a 

(64C) 

P 

•2 

65 

45 

a 

13 

Military  Police 
(Ml) 

M 

81 

66 

S3 

41 

a 

P 

*3 

67 

45 

31 

u 

Petrol  Supply  Specialist 

b 

71 

64 

50 

32 

a 

(76W) 

P 

82 

63 

46 

a 

12 

Aaao  Specialist 

M 

78 

65 

46 

33 

22 

(550) 

P 

84 

72 

55 

37 

19 
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Yeti*  6.4 

Hu  values  of  MOS/Pwfemance  Lml  Cs01»et1***  •*<■!  Sefcjectiv* 
Uttuti  and  Pallid  Ce^arlsm  tat  la  Scallef  Tacbolgae* 

(a  •  12,  Workshops  4  and  7) 


PorferaaiiCk  Percent  111 


m 

Method* 

M- 

JB- 

-Zfi_ 

Jfi. 

Adaialstratlre  Specialist 
(71L) 

SE 

PC 

-.07 

.09 

.29 

.16 

.47 

.24 

.74 

.31 

.86 

.4$ 

/munition  Specialist 
(S5» 

s 

PC 

.12 

.12 

.46 

.26 

.69 

.38 

.90 

.52 

1.13 

.73 

Araar  Crmamn 
(19C/K) 

SE 

PC 

.40 

.25 

.68 

.48 

1.03 

.73 

1.26 

1.14 

1.60 

1.63 

Cannon  Ciwin 
(1)1) 

SE 

PC 

.30 

.24 

.69 

.41 

.93 

.64 

1.24 

.90 

1.49 

1.28 

Car^etrtry/Masonry  Specialist 

SE 

PC 

.00 

.07 

.09 

.11 

.37 

.18 

.61 

.24 

.80 

.38 

Oval  cal  Operations  Specialist 
(54C) 

SE 

PC 

.20 

.16 

.S3 

.35 

.16 

.0 

1.16 

.67 

1.38 

.96 

Coebat  Engineer 
(121) 

SE 

PC 

.20 

.19 

.0 

.36 

.96 

.57 

1.22 

.77 

1.52 

1.0 

Food  Service  Spoclallst 
(9<*) 

SE 

PC 

.09 

.11 

.33 

.18 

.59 

.27 

.0 

.38 

1.04 

.50 

Infantryman 

(118) 

SE 

PC 

.29 

.39 

.71 

.0 

1.00 

1.00 

1.30 

1.53 

1.58 

2.18 

Light  Wheal  Vehicle  Mechanic 
(*3«; 

SE 

n. 

.17 
•  * 

.51 

.0 

1.02 

•SC 

1.24 

.£5 

MAhPADS  C rowan 
(16S) 

SE 

PC 

.19 

.16 

.57 

.31 

.83 

.45 

1.09 

.65 

1.38 

.96 

Medical  Spoclallst 
(918) 

SE 

PC 

.17 

.15 

.48 

.30 

.79 

.42 

1.07 

.62 

1-37 

.95 

Military  Folic* 

(958) 

SE 

PC 

.15 

.16 

.47 

.26 

.71 

.38 

.97 

.52 

1.20 

.74 

Motor  Trans.  Operator 
(64C) 

SE 

PC 

.06 

.13 

.39 

.21 

.59 

.33 

.0 

.43 

.97 

.65 

Petrol.  Supply  Specialist 
(76*0 

SE 

PC 

.16 

.i3 

.51 

.25 

.72 

.39 

.82 

.52 

1.11 

«** 

•  /  O 

Single  Channel  Radio  Operator 
(OSC) 

SE 

PC 

.13 

.16 

.54 

.26 

.77 

.42 

1.09 
•  S3 

1.30 

.80 

TW/Oraoon  Repairer 
(27C) 

SE 

PC 

.10 

.16 

.0 

.28 

.74 

.43 

.99 

.56 

1.33 

.78 

Unit  Supply  Specialist 
(76T) 

SE 

PC 

.08 

.12 

.40 

.22 

.60 

.34 

.91 

-50 

1.07 

.69 

Utility  Helicopter  Repairer 
(67V) 

SE 

PC 

.IS 

.17 

.49 

.30 

.82 

.43 

1.06 

.62 

1.32 

.90 

*  SE:  Slightly  greater  4«c)<m  In  lcuer  half  than  In  upper  for  both  eoebat 
«n4  MXicoafctt. 

K:  SUghtiy  greiter  decline  In  upper  half  than  1w*r  h*lf  for  both 
coebat  And  noncoabat  b«t  soeewhat  larger  for  coebat. 
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subjective  estimation  procedure  are  higher  than  those  obtained  using  the 
paired-comparison  ratio  scaling  technique. 

Examination  of  the  utilities  assigned  to  the  performance  levels  within 
HOS  revealed  that  on  the  average,  for  both  the  combat  and  nonco>nbat  MOS,  the 
subjective  estimation  utility  values  had  a  somewhat  greater  decline  In  the 
lower  half  of  the  performance  levels  (between  the  50th  and  10th  percentiles) 
than  in  the  upper  half  (between  the  90th  and  50th  percentiles).  The  paired* 
comparison  utility  values,  on  the  other  hand,  on  the  average  had  a  somewhat 
greater  decline  in  the  upper  half  of  the  performance  levels  than  in  the 
lower  half  for  both  kinds  of  HOS. 

As  In  the  case  of  the  scenario  differences,  these  scaling  method  dif¬ 
ferences  may  or  may  not  have  practical  significance.  The  correlation  between 
the  mean  values  assigned  the  95  combinations  by  the  two  methods  was  .91. 

It  Is  also  of  Interest  to  note  that  in  general  the  highest  disagreement 
in  assigning  scale  values  occurred  with  high-performance-level  noncoobat  MOS 
combinations,  whereas  the  highest  agreement  in  assigning  scale  values 
occurred  with  low-performance- level  noncoobat  MOS  combinations. 

In  general,  however,  as  noted  earlier,  the  Army  officers  seem  to  have  a 
fairly  common  frame  of  reference.  The  median  intercorrelations  among  the 
officers  for  the  wartime  rank  orders  and  scaling  values  ranged  from  .76  to 
.80.  Average  scale  values  based  upon  the  judgments  of  10  or  more  officers 
should  therefore  have  reliabilities  of  .95  or  higher. 


Perhaps  the  most  significant  finding  is  that  Army  officers  would  be 
willing  and  able  to  assign  differential  utility  values  across  MOS  and 
performance  levels.  Perhaps  the  next  most  significant  finding  Is  that 
fairly  stable  scale  values  could  be  obtained  from  averaging  across  a 
relatively  small  number  of  off leer/ judges. 

In  addition,  the  scenario(s)  used  should  be  free  of  the  detail  that 
suggests  greater  or  less  utility  for  certain  specific  MOS.  Utilities  of 
soldiers  In  wartime  should  not  be  expressed  in  terms  of  dollars;  an  accept¬ 
able  metric  would  be  the  utility  of  a  50th  percentile  Infantryman  (his 
value  for  the  survival  of  the  unit  and  In  replacing  troop  losses  is  much 
more  readily  apparent).  Directions  to  the  judges  should  be  reassuring 
concerning  inconsistencies  that  can  possibly  occur  in  a  long  series  of 
judgments. 

As  discussed  earlier,  some  of  the  problems  identified  (e.g.,  scenario 
effects)  may  have  little  practical  significance  in  terms  of  how  a  computer¬ 
ized  enlisted  personnel  selection  and  classification  system  woc’d  process 
Army  applicants  under  operational  constraints.  Further  research  should 
examine,  through  sensitivity  analyses  and  computer  simulations,  how  dif¬ 
ferences  in  the  utilities  of  MOS/performance  level  combinations  affect 
system  output. 


102 


TRYOUT  OF  METHODS  FOR  ASSIGNING  UTILITIES  TO  PERFORMANCE  LEVELS 


The  second  phase  of  utility  scale  development  was  devoted  to  formula- 
tlng  the  final  'procedures  to  be  used  In  actually  assigning  utilities  to 
performance  levels  In  all  entry-level  HOS.  Several  Inferences  drawn  from 
the  exploratory  findings  In  the  earlier  workshops  guided  the  developmental 
process. 

First,  the  apparent  nonlinear  relationships  between  utility  and 
performance  found  In  some  HOS  would  necessitate  obtaining  judgnents  of  the 
utility  of  at  least  five  performance  levels  within  each  HOS.  Five  data 
points  would  allow  the  derivation  of  a  best  fitting  utility/performance 
curve  with  two  Inflection  points  (if  necessary)  within  an  HOS. 

Second,  the  task  of  assigning  utility  scale  values  to  at  least  five 
performance  levels  In  275  MOS  was  ouch  too  onerous  to  assign  to  any  one 
judge.  Some  system  for  obtaining  the  judgments  would  need  to  be  employed 
that  allowed  the  task  to  be  divided  among  groups  of  judges,  but  that  still 
allowed  utilities  to  be  reliably  scaled  both  between  and  within  HOS. 

Third,  the  system  used  to  obtain  judgments  from  a  group  of  judges  could 
employ  more  than  one  scaling  method.  The  high  correlations  between  utility 
values  obtained  earlier  from  different  scaling  methods  suggested  that  a 
combination  of  methods  might  allow  the  overall  scaling  task  to  be  accom¬ 
plished  more  efficiently  than  through  using  one  method  only.  The  goal  was 
to  place  on  the  same  ratio  scale  the  utility  values  of  at  least  275  x  5,  or 
1,375,  MOS/performance  Level/combinations.  (A  ratio  scale  would  permit 
utilities  to  be  summed  across  individual  MOS  assignments  in  comparing 
se lect lon/c 1 ass 1 f i cation  systems . ) 

Procedure 

An  additional  12  workshops  ware  conducted  to  try  out  various  scaling 
methods  and  to  prepare  for  the  third  phase  of  the  research,  in  which  the 
selected  scaling  methods  would  be  applied  to  all  entry-level  MOS  and  within- 
MOS  performance  levels.  These  workshops,  like  the  previous  ones,  were 
attended  by  small  groups  of  Army  field  grade  officers. 

The  methods  tried  out  at  the  workshops  were  rank  ordering,  paired 
comparisons,  a  conjoint  scaling  procedure,  the  sorting  or  placement  of 
MOS/performance  level  combinations  into  piles,  and  the  direct  estimation  of 
ratio  scale  values  using  a  standard  MOS/performance  level  set  at  100.  Of 
these  techniques,  the  latter  two  were  the  scaling  procedures  eventually 
selected. 

Alternative  Methods 

The  rank  ordering  task  in"olved  rank  ordering  a  list  of  135  MOS  with 
all  performance  levels  set  at  the  50th  percentile.  This  method  produced 
negative  reactions  from  the  workshop  participants.  They  objected  to  the 
time  it  took  to  perform  the  rankings  and  to  t.  jir  inability  to  assign  tied 
ranks  under  the  method  used.  They  felt  that  they  did  not  know  enough  about 


*11  the  MGS  to  sake  the  fine  discriminations  called  for  in  rank  ordering. 
They  also  objected  to  the  very  task  of  tank  ordering  HOS.  saying  that  til 
Army  MOS  were  important.  Though  the  Project  A  staff  had  anticipated  this 
latter  reaction  from  some  officers  in  earlier  workshops,  It  was  not 
exhibited  until  this  instance,  in  which  participsr  ts  were  asked  to  make 
utility  judgments  only  between  MOS,  since  all  performance  levels  were  set  at 
the  50th  percentile. 

Modifications  to  deal  with  four  Issues  were  incorporated  in  the 
subsequent  workshops.  First,  it  was  decided  to  use  only  scaling  methods 
that  allowed  judges  to  report  that  two  or  more  MOS  performance  levels 
combinations  were  approximately  equal  in  utility.  Second,  the  judges  were 
offered  the  alternative  of  not  evaluating  the  utility  of  some  MOS  if  they 
felt  they  did  not  know  enough  abaut  the  MOS  to  make  informed  judgments. 
Third,  different  performance  levels  were  included  within  MOS,  as  well  as 
between  MOS,  In  the  set  of  combinations  to  be  judged. 

The  fourth  change  involved  placing  the  judgments  in  a  selection  and 
classification  context.  That  is,  the  officers  were  asked  to  judge  the 
utility  of  predicted  pen'ormance  of  Army  applicants  or  recruits  rather  than 
actual  performance  of  Army  job  Incumbents  (as  had  been  done  in  the  earlier 
workshops).  Percentile  levels  were  still  used  as  In  earlier  workshops,  but 
the  percentiles  were  for  predicted  performance  for  the  given  MOS  of  ill  Army 
applicants  or  recruits.  The  judges  were  asked  to  assume  that  the  perfor¬ 
mance  percentiles  given  were  accurate  estimates  of  future  on-the-job 
performance  percentiles  if  the  applicants  or  recruits  were  actually  assigned 
to  the  MOS.  After  this  adjustment  was  made,  none  of  the  judges  in  subse¬ 
quent  workshops  objected  to  the  basic  concept  o*  assigning  differential 
utilities  to  various  MOS/ performance  levels. 

Two  variants  of  the  paired-comparison  method  were  tried  out,  using  a 
limited  number  of  MOS/perforaance  level  combinations.  One  involved  judg¬ 
ments  of  number  equality,  as  in  Workshops  4  and  5.  The  other  involved 
assigning  100  enlisted  applicants  with  given  predicted  performance  percen¬ 
tiles  to  pairs  of  MOS;  for  example,  if  there  were  100  applicants  who  were  at 
the  10th  percentile  for  the  job  of  illustrator  (MOS  81E),  and  at  the  30th 
percentile  for  the  job  of  physical  activities  specialist  (MOS  03C),  how  many 
of  the  100  applicants  should  be  assigned  to  each  job? 

Though  both  of  these  paired-comparison  tasks  called  for  complex 
judgments,  the  officers  performed  thee  readily.  However,  the  methodology 
was  time  consuming,  and  would  be  even  more  so  with  larger  numbers  of 
MQS/performance  level  combinations  to  judge.  Moreover,  the  officers  felt 
they  should  be  allowed  to  indicate  that  some  applicants  should  not  be 
selected  at  all.  The  judgment  was  subsequently  shifted  from  predicted 
performance  levels  of  applicants  to  that  of  recruits  (selected  applicants), 
thereby  eliminating  the  "do  not  select*  alternative.  However,  the  judges 
were  allowed  to  indicate  that  they  thought  a  given  recruit  would  have  zero 
or  negative  utility  for  the  Array  if  placed  in  an  MOS  where  his  or  her 
predicted  performance  was  low. 


Other  questions  that  were  raised  concerned  the  field  strength  of  A  nay 
units  staffed  with  various  MOS  complements,  and  the  possibility  of  potential 
troop  losses  If  open  warfare  broke  out.  Some  officers  reported  that  they 
considered  these  factors  In  evaluating  the  utility  of  the  applicants  or 
recruits,  (o  divorce  both  troop  strength  and  troop  replacements  from 
utility/assignment  decisions,  judges  in  subsequent  workshops  were  told  that 
the  field  strength  of  all  MOS  was  70  percent  and  that  the  problem  o; 
compensating  for  troop  losses  was  being  handled  by  another  pari  of  the 
assignment  algorithm  and  should  not  enter  into  their  MOS  utility  judgments. 

A  conjoint  scaling  method  was  also  tried  out  to  determine  whether  It 
was  possible  to  obtair  MOS/performance  level  utility  evaluations  at  the  same 
time  that  weights  wert  established  for  different  components  of  performance. 
Each  of  16  MOS  was  paired  with  each  other  MOS  in  the  set,  at  the  same  time 
that  predicted  percentile  levels  for  15  different  pairs  of  performance 
factors  were  given.  Although  a  conjoint  procedure  later  proved  effective 
for  use  In  arriving  at  weights  for  combining  performance  factors  into 
overall  measures  of  MOS  performance  (see  Chapter  3),  the  method  tried  here 
was  much  too  difficult  and  time  consuming  for  use  in  scaling  large  nuciu-  rs 
of  MOS/performance  level  combinations. 

One  method  that  did  prove  effective  for  making  large  numbers  of  scaling 
utility  decisions  was  the  pile  placement  method,  in  which  judges  sorted 
cards  containing  MOS/performance  level  combinations  into  piles,  based  upon 
their  perceived  utility  or  selection  priority.  Seven  piles  of  predicted 
performance  utility  were  used,  ranging  from  negative  through  zero  utilit>  to, 
high  utility.  The  judges  initially  sorted  135  MOS/performance  level 
combinations,  then  210  combinations,  and  eventually  280  combinations, 
without  complaining  about  the  judgment  burden. 

Likewise,  judges  in  the  ratio  judgment  method,  in  which  they  ev:  *:«ated 
MOS/performance  level  utilities  in  relationship  to  that  of  a  90th  percentile 
Infantryman,  judged  59  combinations  without  the  task  becoming  burder<-cme. 

Using  data  from  one  of  the  last  workshops  in  phase  two,  separate 
analyses  of  variance  were  performed  on  the  mean  pile  placements  and  ratio 
judgments  given  59  combinations  judged  by  the  12  officers  using  both 
methods.  Remarkably  similar  £  ratio  patterns  were  obtained  (see  Table  4.5). 
Tor  both  scaling  methods,  there  were  large  mean  differences  in  assigned 
ratings  for  different  predicted  performance  levels.  Likewise,  there  were 
significant  mean  differences  for  rater  and  MOS,  while  the  MOS  by  Percentile 
level  interaction  was  not  significant  in  either  analysis.  The  intraclass 
correlation  reliability  estimate  for  the  pile  placement  procedure  was  .58 
and  the  comparable  coefficient  for  the  direct  ratio  judgment  was  .65. 

These  results  indicated  that  satisfactory  reliabilities  for  mean 
utilities  could  be  obtained  by  both  methods,  if  the  means  were  based  upon  10 
or  more  judges.  The  correlation  between  the  mean  utilities  assigned  by  the 
12  officers  to  the  59  conmon  combinations  using  the  two  methods  was  .89. 
Though  this  Intermethod  correlation  was  not  as  high  as  might  be  desired,  it 
should  be  increased  by  using  MOS/perfotmance  level  combinations  with  a 
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Table  4.5 

Analysis  of  Variance  of  Pile  PUcemant  and  Ratio  Utility  Valuas, 
Based  on  59  Cocnon  Combinations  (n  ■  12  Officers,  Workshop  11) 


Pile 

Placement 

Ratio 

Source 

df 

P>  F 

F 

J>1 L 

Model 

69 

14.76 

.0001 

.61 

23.54 

.0001 

.72 

Error 

638 

TOTAL 

707 

MOS 

11 

1.91 

.0358 

10.98 

.0001 

Level 

4 

220.37 

.0001 

319.42 

.0001 

MOS  X  Level 

43 

.61 

.9771 

.90 

.6589 

Rater 

11 

8.17 

.0001 

17.01 

.0001 

greater  range  of  utilities  and  by  raising  the  reliability  of  both  sets  of 
scale  values  by  an  increase  in  the  number  of  judges. 

Methods  .Selected 

In  the  light  of  all  the  information  available  from  the  first  and  second 
phase  workshops,  it  was  decided  to  use  the  pile  placement  and  direct  ratio 
estimation  methods  in  the  final  determinations  of  the  utilities  of  approxi¬ 
mately  275  MOS  x  5  performance  levels,  or  1,375  combinations.  The  pile 
placement  method  provided  a  means  of  reliably  scaling  the  utility  of  large 
numbers  of  combinations  in  a  reasonable  time  period,  while  the  direct 
estimation  method  could  be  used  to  place  a  limited  number  of  combinations  on 
a  reliable  ratio  scale  having  a  meaningful  zero  point  and  the  potential  for 
assigning  negative  utilities  to  low  predicted  performance  levels. 

The  procedures  used  to  place  the  1,375  combinations  on  one  utility 
scale  are  described  in  the  next  section,  which  also  presents  the  results  of 
the  third  and  final  phase  of  the  utility  scaling  effort. 


OBTAINING  A  COMPLETE  SET  OF  UlILITY  ESTIMATES 

The  goal  of  the  exploratory  workshops  was  to  develop  a  scaling 
metbod(s)  for  obtaining  utility  functions,  for  a  large  set  of  MOS,  that  to 
the  maximum  extent  possible  reflected  the  relative  payoff  to  the  Army  of 
different  levels  of  job  performance.  The  results  of  these  exploratory 
workshops  were  largely  successful.  First,  for  the  Army  jobs  being  consid¬ 
ered  in  these  workshops,  utility  scale  values  varied  across  MOS  in  a  manner 
generally  consistent  with  expectations.  Second,  the  utility  values  assigned 
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by  the  officers  were  sufficiently  alike  to  indicate  that  fairly  stable  scale 
values  could  be  obtained  by  averaging  across  officer  judgments.  Collec¬ 
tively,  these  results  pointed  to  the  feasibility  of  obtaining  information  on 
the  relative  value  of  performance  in  Army  MOS  that  could  be  used  to  guide 
decisions  in  a  personnel  selection  and  classification  system. 

The  next  goal  was  to  assign  a  utility  to  any  predicted  level  of  perfor¬ 
mance  on  any  entry-level  MOS.  The  obtained  utility  values  could  be  used  to 
(a)  assess  the  net  gain  to  the  Army  of  using  new  selection/classification 
procedures,  and  (b)  help  guide  classification  algorithms  in  optimizing 
assignments  of  individual  recruits. 

Method 


Design  Issues.  Observations  made  in  earlier  workshops  of  the  amount  of 
time  it  took  field  grade  officers  to  place  MOS/performance  level  combina¬ 
tions  in  piles  on  the  basis  of  judged  utility  Indicated  that  they  could 
judge  250  combinations  in  about  1  1/2  hours.  Similar  observations  of  the 
amount  of  time  It  took  to  directly  judge  the  utility  of  combinations 
relative  to  the  standard  of  a  90th  percentile  Infantryman  indicated  that  the 
officers  could  judge  50  combinations  in  about  40  minutes.  It  was  apparent 
that  only  a  subset  of  the  total  number  of  MOS/performance  level  combinations 
could  be  presented  to  any  one  officer. 

To  place  all  utilities  on  a  ratio  scale,  the  project  staff  chose  to  use 
both  the  pile  placement  method  and  a  direct  judgment  method.  The  pile 
placement  method  would  be  used  to  place  the  utilities  on  an  Interval  scale 
and  the  direct  estimation  method  would  be  used  to  develop  a  ratio  scale  tor 
a  target  set  of  MOS  by  performance  combinations  against  which  the  interval 
scale  values  could  be  calibrated. 

To  cierge  the  utilities  obtained  from  the  two  methods,  the  same  officers 
should  judge  a  common  set  of  MOS/performance  level  combinations  using  both 
methods.  Therefore,  to  adjust  for  differences  among  the  samples  of  officers 
assigned  particular  subsets  of  MOS/performance  level  combinations  In  the 
pile  sorting  method,  all  judges  were  asked  to  judge  one  common  set  of  60 
combinations  using  both  methods. 

Another  issue  concerned  the  number  of  performance  levels  within  each 
MOS.  Because  of  the  large  number  of  entry-level  MOS,  the  number  of  perfor¬ 
mance  levels  within  each  MOS  was  restricted  to  five.  This  number  still 
allowed  for  the  derivation  of  a  nonlinear  function  with  Inflection  points 
when  expressing  utility  as  a  function  of  the  level  of  performance  within 
each  MOS. 

SeUction  and  Grouping  of  Entry-level  MOS.  AR  611-201,  Enlisted  Career 
Management  Fields  and  Military  Occupational  Specialties  (October  1985),  was 
used  in  the  selection  of  276  entry-level  MOS.  All  MOS  that  listed  Skill 
Level  1  duties  and  that  required  an  ASVAB  Aptitude  Area  score  for  assignment 
to  the  MOS  were  selected. 
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Consequently,  there  were  276  MOS  times  5  levels  or  1,380  MOS/perfor- 
a&nce  level  combinations  to  be  judged  separately.  To  sake  the  scaling  task 
more  acceptable  to  the  judges,  seven  sets  of  combinations  were  formed.  The 
first  set,  consisting  of  12  HOS  times  5  performance  levels,  or  60  combina¬ 
tions,  was  to  be  judged  by  all  judges  and  would  provide  the  basis  for  a 
common  utility  scale. 

The  Infantryaan  (118)  MOS  was  selected  first  because  the  utility  of  the 
90th  percentile  infantryman  was  to  be  used  as  the  standard  (set  at  100) 
against  which  the  utility  of  all  other  MOS/performance  level  combinations 
was  to  be  compared  in  the  direct  judgment  method.  Judgment  data  from 
earlier  workshops  were  used  to  identify  11  additional  MOS  that  met  the 
following  three  criteria;  (a)  Ho  officers  had  refused  to  scale  the  MOS 
because  of  unfamiliarity  with  the  MOS;  (b)  utility  values  for  the  55 
MOS/performance  level  combinations  were  evenly  spread  across  the  ranae  of 
utilities  assigned  all  MOS  used  in  the  workshops;  and  (c)  extremely  low  or 
negative  utility  values  were  likely  to  be  obtained  for  performance  in  some 
of  the  MOS  at  the  10th  percentile  and  high  utility  values  were  expected  for 
some  jobs  at  the  90th  percentile. 

The  remaining  264  MOS  were  grouped  into  six  subsets  of  44  MOS  each. 

The  subsets  were  made  comparable  through  a  systematic,  stratified  assignment 
procedure.  The  MOS  were  first  grouped  in  accordance  with  the  results  of  a 
cluster  analysis  based  upon  judgments  of  job  and  task  similarity  (Hoffman, 
1987).  This  analysis  identified  23  MOS  clusters.  The  MOS  in  each  cluster 
were  placed  jn  numerical-alphabetical  sequence.  Then  every  sixth  MOS  was 
assigned  to  one  of  the  six  subsets,  which  were  labeled  Decks  A,  B,  C,  D,  E, 
and  F. 


Each  subset  or  deck  contained  280  MOS/performance  level  coobinations  — 
12  common  MOS  plus  44  noncomoon  MOS  and  five  performance  levels  for  each 
job2.  The  combinations  in  each  deck  were  randomized  before  being  adminis¬ 
tered  to  the  judges. 


Sample  of  Officers  Used  as  Judges.  Data  from  the  exploratory  workshops 
indicated  that  about  10  officers  would  be  needed  to  obtain  an  inter judge 
reliability  of  about  .95  in  utility  judgments.  To  ensure  that  a  total 
sample  of  60  officers  (10  officers  x  6  decks)  was  obtained,  utility  work¬ 
shops  were  held  at  six  CONUS  Army  pests  and  in  "SAPEUR.  Altogether,  74 
field  grade  officers  attended  the  workshops.  The  74  participants  consisted 
of  14  Infantry,  21  Armor,  14  Other  Combat  (e.g.,  Artillery),  12  Combat 
Support,  and  13  Combat  Service  Support  officers.  Most  of  the  officers  were 
majors;  there  were  54  majors,  13  lieutenant  colonels,  and  7  colonels  among 
the  participants. 


2Three  noncommon  MOS  and  their  performance  levels  were  subsequently 
dropped  from  the  judgment  sets  because  the  three  MOS  were  rescinded  from 
operational  status.  Two  of  these  MOS  were  in  Deck  C,  the  other  in  Deck  E. 
The  number  of  combinations  in  the  six  decks  therefore  ranged  from  270  to 
280.  For  convenience,  the  number  280  is  used  in  the  text  and  tables  to 
Indicate  the  number  of  MOS/performance  level  combinations  in  the  decks. 
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The  Utility  Judgment  Workshops 


After  a  brief  overview  of  Project  A  and  description  of  the  workshop 
agenda  by  the  workshop  leader,  the  participants  completed  a  Background 
Information  Sheet,  including  items  pertaining  to  arade,  military  specialty, 
current  and  previous  positions,  and  years  of  service. 

The  leader  then  gave  a  more  detailed  overview  of  the  workshop  and  its 
purpose,  and  discussed  assumptions  that  the  participants  were  to  use  in 
making  their  judgments.  The  three  critical  assumptions  are  given  below  (the 
complete  set  of  assumptions  and  copies  of  the  workshop  instructions  and 
forms  are  supplied  in  Appendix  0). 

(1)  The  military  context  for  which  the  utility  of  the  recruits  is 
being  considered  is  as  follows:  The  world  is  in  a  period  of 
heightened  tensions.  There  is  an  increasing  probability  that 
hostilities  will  break  out  in  Europe,  Asia,  the  Caribbean,  Latin 
America,  and  Africa.  The  Army's  mission  is  to  support  U.S.  treaty 
obligations  and  to  help  defend  the  borders  of  allied  and  friendly 
nations.  Some  of  the  potential  enemies  have  nuclear  and  chemical 
capability.  Air  parity  does  exist  between  allied  forces  and 
potential  hostile  nations.  U.S.  Army  training  and  other  prepara¬ 
tory  activities  have  been  substantially  increased.  Host  combat 
and  associated  support  units  are  participating  in  frequent  field 
exercises.  Host  units  are  being  actively  resupplied. 

(2)  The  overall  HOS  performance  measure  for  each  HOS  represents  an 
optimally  weighted  (for  that  HOS)  cofiibination  of  several  perfor¬ 
mance  factors.  Thus,  recruits  at  the  highest  predicted  perfor¬ 
mance  level  (90th  percentile)  in  each  MOS  are  more  likely  to  be 
dependable,  be  proficient  in  HOS  tasks,  know  the  facts  and 
procedures  required  to  do  their  jobs,  perform  more  effectively 
under  adverse  or  difficult  condit.  .'s,  avoid  disciplinary  prob¬ 
lems,  provide  support  to  fellow  soldiers,  and  be  more  physically 
fit. 

(3)  The  predicted  performance  levels  for  the  recruits  are  accurate. 
That  is,  the  recruits  will  actually  perform  at  the  predicted 
levels. 

Note  that  the  judgment  called  for  in  both  utility  scaling  methods  was 
the  relative  value  of  recruits  with  different  predicted  performance  levels 
for  the  entry-level  HOS.  The  decision  to  use  predicted  performance  levels 
rather  than  actual  performance  levels  was  based  upon  the  fact  that,  in 
application,  a  computerized  enlisted  personnel  selection  and  classification 
system  would  be  operating  with  predicted  or  estimated  performance  (a  major 
purpose  of  Project  A  is  to  improve  the  accuracy  of  that  prediction). 

The  participants  then  read  a  description  of  the  pile  placement  method. 
Emphasis  was  placed  on  the  definitions  given  the  seven  piles  in  which  the 
judges  were  to  place  the  HOS/ performance  level  combinations. 
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•  High  positive  utility  would  probably  result  If  these  recruits 
were  placed  In  these  MOS. 

t  Between  moderate  and  high  utility  would  probably  result  If  these 
recruits  were  placed  in  these  MOS. 

e  Moderate  utility  would  probably  result  If  these  recruits  were 
placed  in  these  MOS. 

•  Between  low  and  moderate  utility  would  probably  result  If  these 
recruits  were  placed  In  these  MOS. 

•  Low  positive  utility  would  probably  result  if  these  recruits  were 
placed  in  these  MOS. 

t  Advantages  of  placing  these  recruits  In  these  MOS  would  probably 
be  equal  to  the  disadvantages  (expected  utility  »  0). 

e  Negative  utility  would  probably  result  if  these  recruits  were 
placed  in  these  MOS.  (Any  positive  contribution  would  probably 
be  outweighed  by  problems  associated  with  low  levels  of  overall 
performance. ) 

Each  MOS/perf ortnance  level  combination  was  printed  on  a  separate  card 
(see  samples  in  Figure  4.3).  On  each  card  there  was  a  short  description  of 
the  MOS  (Skill  Level  1)  taken  from  AR  611-2Q1.  The  performance  level, 
either  the  10th c  30th,  50th,  70th,  or  90th  percentile,  was  also  listed  on 
the  card.  The  instructions  indicated  that  the  percentiles  were  the  predic¬ 
ted  performance  percentiles  of  recruits,  if  all  recruits  were  rank  ordered 
in  terms  of  their  predicted  performance  in  the  given  MOS  without  regard  to 
current  cut-off  scores.  The  instructions  also  allowed  the  judges  to  place 
in  an  eighth  unrated  pile  any  MQS/performance  level  combinations  that  they 
were  not  familiar  enough  with  to  evaluate.  No  restrictions  were  placed  on 
the  number  of  cards  that  could  be  placed  In  any  one  pile.  The  cards  in  each 
deck  were  thoroughly  shuffled  prior  to  the  workshops.  Decks  were  assigned 
to  participants  randomly. 

Upon  completing  the  pile  placement  method  and  a  short  break,  the  judges 
read  the  instructions  for  the  direct  judgment  method.  After  first  reviewing 
the  assumptions  and  re-familiarizing  themselves  with  the  12  cuiiriCm  MOS,  the 
participants  wrote  the  value,  100,  on  the  90th  percentile  Infantryman  card, 
which  was  on  top  cf  the  deck  of  60  cards.  The  task  of  the  judges  was  to 
assign  a  utility  value  to  each  of  the  remaining  59  MOS/performance  level 
combinations,  taking  Into  consideration  that  a  90th  percentile  infantryman 
had  a  utility  of  100.  Zero  and  negative  utility  values  were  permitted. 

The  judges  wrote  the  assigned  utilities  directly  on  the  cards.  After 
they  had  gone  through  the  deck  once,  they  were  instructed  to  arrange  the 
cards  in  ascending  numerical  order  and  then  go  through  the  cards  again, 
changing  any  utility  values  that  they  felt  were  out  of  line  v/ith  the  others 
in  terms  of  the  ratios  of  the  assigned  utilities. 
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RADIO  TELETYPE  OPERATOR 
(Radio  TT  Operator)  (MOS  05C) 

SUMMARY:  Supervises  or  operates  and  installs  radio  teletypewriter 
and  tape  relay  equipment  in  radio  teletypewriter  and  tape  relay  tactical 
or  administrative  communications  nets. 

DUTIES:  Operates  radio  teletype  equipment  to  transmit  and  receive 
messages. 

OVERALL  EFFECTIVENESS:  70  percentile 


MLAYY  AMTIARMOR  WEAPONS  INFANTRYMAN 
(MOS11H) 

SUMMARY:  Leads  or  serves  as  member  of  heavy  antiarmor  crew- 
served  weapons  squad-,  section,  or  platoon  employing  heavy  antiarmor 
crew-served  weapons  in  offensive  and  defensive  combat  operations. 

DUTIES:  Assaults  and  destroys  enemy  tanks  and  armor  vehicles, 
emplacements,  weapons,  and  personnel  with  heavy  antiarmor  weapons 
(TOW). 

OVERALL  EFFECTIVENESS:  30  percentile 


Figure  4.3.  S&aples  of  MOS/Perforaance  Level  combination  cards. 
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Reliability  and  Validity  Analyses 

Identification  and  Delation  of  Atypical  Judaas.  An  Initial  question 
posed  was  whether  sons  of  the  Judges'  responses  were  sufficiently  atypical 
on  a  priori  grounds  to  warrant  excluding  these  judaes  from  later  analyses. 
For  example,  If  any  of  the  participants  did  not  fully  comprehend  the  task  or 
its  underlying  assumptions,  or  If  they  were  inattentive  in  accomplishing  the 
task,  then  inclusion  of  their  data  could  decrease  the  reliability  and 
validity  of  the  final  scale  values. 

Four  indexes  were  used  to  determine  the  degree  of  atypicality: 

(1)  The  number  of  times  a  judge  assigned  greater  utilities  to  lower 
performance  level  recruits  than  to  higher  performance  level 
recruits  in  the  same  MOS. 

(2)  The  median  correlation  of  the  utilities  assigned  by  a  judge  across 
the  MOS/perfcrmance  level  combinations  with  the  utilities  assigned 
by  the  other  judges. 

(3)  The  mean  utility  assigned  the  MOS/performance  level  combinations 
by  a  judge.  Unusually  high  or  low  mean  values  would  indicate  that 
the  judge  was  assigning  many  of  the  combinations  greater  or  lesser 
utilities  than  the  other  judges  were. 

(4)  The  standard  deviation  of  the  utilities  assigned  the  MOS/perfor- 
aance  level  combinations  by  a  judge.  Large  or  small  standard 
deviations  would  indicate  that  the  judge  was  assigning  an  unusual¬ 
ly  wide  or  narrow  range  of  utilities  to  the  combinations. 

These  indexes  were  calculated  for  both  the  pile  placement  and  the  direct 
judgment  data  from  each  judge.  Frequency  distributions  of  the  eight  sets  of 
indexes  were  examined  and  the  judges  who  had  relatively  extreme  values  were 
identified. 

Of  the  eight  indexes,  those  considered  to  be  most  indicative  of 
atypicality  were  high  numbers  of  inversions  and/or  low  median  correlations 
with  other  judges  for  either  the  pile  placement  or  the  direct  judgment  data. 
A  "rule  of  thumb1'  was  adopted  that  a  judge  nan  to  have  atypical  values  on  at 
least  one  of  these  four  indexes  to  be  considered  for  exclusion.  On  the 
basis  of  this  rule,  seven  judges  were  excluded.  Tables  4.6  and  4.7  show  the 
frequency  distributions  for  the  number  of  within-MOS  inversions.  Table  4.8 
shows  the  frequency  distributions  of  the  median  correlations  between  the 
judges. 

The  median  correlations  for  the  pile  placement  data  were  obtained  by 
forming  a  separate  intercorrelation  matrix  for  the  judges  assigned  each 
deck.  Here  the  correlation  between  any  pair  of  judges  was  computed  across 
their  joint  pile  placements  of  the  280  combinations  in  their  respective 
decks.  The  median  correlations  for  the  direct  judgment  data  were  obtained 
by  forming  an  intercorrelation  matrix  for  all  judges.  The  correlation 
between  any  two  judges  was  computed  across  the  utility  values  they  assigned 
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Table  4.6 


Frequency  Distribution  of  Number  of  Flit  Placement  Inversions  Hade  by  Judges 
(280  Combinations) 


Number  of 

Number  of 

Inversions 

Judaes 

0 

12 

1  -  10 

24 

11-20 

10 

21-30 

8 

31-40 

7 

41  -  50 

2 

51  -  60 

2 

61  -  70 

2 

71-80 

1 

81  -  100 

0 

101  -  200 

4 

4,  38,  64.  84 

201  and  Above 

2 

67.  83 

Total 

74 

a  Some  Identification  numbers  for  Judges  are  higher  than  74  because  earlier 
lists  of  judges  included  a  few  company  grade  officers.  The  final  set  of 
analyses  was  limited  to  field  grade  officers. 


Table  4.7 

Frequency  Distribution  of  Number  of  Direct  Judgment  Inversions  Made  by 
Judges  (60  Common  Combinations) 


Number  of  Number  of 

Inversions  Judges  Judge  ID 


0  23 

1-5  33 

6-10  8 

11-15  5 

16-20  3 

21-40  0 

41  and  Above  _ 2 

Total  74 


4,  53 


Tibia  4.8 

Frequency  Distribution  of  Nedlan  Correlations  Between  Judgas 


Pile  Placement 

Direct  Judgment 

(280  Combinations) 

(60  Common  Combinations) 

Median 

Correlation 

No.  of 

No.  cf 

With  Other  Judges 

Judges 

Judge  ID 

judges 

AtsiqgJLE 

0  -  .10 

1 

56 

0 

.11  -  .20 

2 

64,  83 

1 

64 

.21  -  .30 

2 

38,  67 

1 

53 

.31  -  .40 

0 

1 

67 

.41  -  .50 

0 

1 

83 

.51  -  .60 

3 

4 

.61  and  Above 

M 

Total 

74 

74 

the  60  common  combinations.  While  over  90  percentile  of  the  judges  had 
median  correlations  above  .50  for  both  types  of  judgments,  six  judges 
had  median  correlations  of  .50  or  below  for  either  the  pile  placement  or  the 
direct  judgment  method.  Three  of  these  judges  had  median  correlations  of 
.50  or  below  for  both  judgment  sets. 

The  frequency  distribution  of  the  mean  pile  placements  did  not  indicate 
that  any  judges  had  mean  pile  placements  that  were  out  of  line  or  atypical. 
However,  for  the  direct  judgments,  the  means  of  the  values  assigned  the  60 
comnon  combinations  by  three  of  the  judges  were  considerably  higher,  and 
those  assigned  by  one  judge  were  considerably  lower,  than  the  mean  values 
assigned  by  the  rest  of  the  judges.  Examination  of  the  frequency  distribu¬ 
tions  of  the  standard  deviations  of  the  pile  placement  and  direct  judgment 
utilities  assigned  by  the  judges  indicated  that  only  three  judges  had  fairly 
atypical  indexes. 


.  »  .  i  . 


iaoie  sunmar i zes  the  iiiTOf uioiion  presented  above  by  shewing 
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judges  who  had  atypical  values  for  one  or  more  of  the  eight  indexes  used. 

Of  the  11  judges  listed,  six  had  two  or  more  atypical  values  for  number  of 
inversions  and/or  median  correlations  with  other  judges.  Using  the  rule  of 
thumb  adopted  earlier,  these  six  judges  were  removed  from  the  sample.  A 
seventh  judge,  identification  number  56,  was  also  removed  because  this 
judge  had  a  median  pile  placement  correlation  with  other  judges  of  .05,  the 
lowest  recorded  for  any  judge. 


Before  removing  these  seven  judges,  a  check  was  made  to  see  whether  as 
a  group  they  were  in  basic  agreement  with  one  another.  (Here  we  were  trying 
to  avoid  possibly  eliminating  a  coherent  minority  of  judges  who  simply  had  a 
different  point  of  view  concerning  the  utility  of  the  various  combinations 
than  did  the  majority  of  field  grade  officers.)  The  intercorrelations 
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Table  4.9 

Judgas  With  One  or  Kora  Problems  in  Thair  Utility  Judgmental  Data 


No.  of 

Median 

Inversions15 

Correlations _ 

Mean  Utilitv 

Utilitv  SD 

Judge 

Pile  PL  filreci 

ID  a 

?,US  £k 

EHU»L 

Direct 

LUl.PL  JUiecl 

*  4 

X  X 

X 

X 

16 

X 

*38 

X 

X 

X 

45 

X 

*53 

X 

X 

*56 

X 

*64 

X 

X 

X 

X 

*67 

X 

X 

X 

*83 

X 

X 

X 

84 

X 

86 

X 

j*  *  indicates  judge's  dat.a  were  removed  from  later  analysesT 
®  Pile  PL  •  Pile  Placement;  Direct  •  Direct  Judgment. 


across  the  60  coonon  combinations  among  the  seven  judges'  utilities  were 
calculated  for  both  the  pile  placement  and  the  direct  judgment  meinour. 

Their  median  intercorrelations  were  .15  for  the  pile  placements  and  .30  for 
the  direct  judgments  (the  remaining  67  judges  had  median  intercorrelations 
of  .50  and  above  for  both  the  pile  placement  and  direct  judgment  methods). 
That  is,  the  seven  judges  agreed  neither  with  the  other  judges  nor  among 
themselves.  Consequently,  their  judgments  were  excluded  from  the  final 
analyses. 

Imputation  of  Kissing  Oata.  A  number  of  the  judges  had  failed  to 
provide  utilities  for  all  the  MOS/performance  level  combinations  they  had 
been  assigned.  Four  judges  did  not  record  their  direct  judgment  utilities 
for  one  or  tv.o  comb: nations.  A  larger  number,  23,  did  not  place  one  or  more 
combinations  in  any  of  the  seven  utility  level  piles  because  they  were  not 
familiar  enough  with  the  job  to  assign  a  utility  value.  Table  4.10  shows  a 
frequency  distribution  by  number  of  unsorted  combinations. 

Although  the  missing  judgments  constituted  only  about  1  percent  of  the 
total  data  set,  there  was  some  concern  that  the  average  scale  values  of  some 
MOS  might  be  unduly  affected  by  not  being  based  on  the  same  set  of  judges  as 
other  HOS.  Consequently,  the  missing  values  were  imputed,  using  a  multiple 
regression  procedure.  Treating  each  judge  with  missing  data  as  a  dependent 
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Table  4.10 


Frequency  Distribution  of  Unsorted  MOS/Performance  Level  Combinations 
(Pile  Placement  Method) 

Number  of 

Unsorted 

Number  of 

Judaes 

0 

51 

1  -  10 

15 

11  -  20 

2 

21  -  30 

4 

31  or  More 

__2 

Total 

74 

variable  and  judges  with  complete  data  as  candidate  independent  variables^, 
analysts  used  a  stepwise  variable  selection  routine  to  select  judges  whose 
utility  ratings  or  pile  placements  added  the  most  to  the  prediction  of  the 
known  utility  values  of  the  judges  with  missing  data.  Judges  (independent 
variables)  were  allowed  to  enter  the  multiple  regression  equation  provided 
that  their  £  ratio  to  enter  was  significant  at  the  .10  level.  The  multiple 
correlation  coefficients  obtained  for  the  most  part  were  .90  or  higher. 

Comparison  of  Nonedlted  and  Edited  jiata.  The  cumulative  effect  of 
removing  seven  judges  and  imputing  pile  placement  or  direct  judgment 
utilities  for  other  judges  was  assessed  through  comparing  the  intercorrela¬ 
tions  and  reliabilities  of  the  nonedited  and  edited  data  sets.  Table  4.11 
shows  that  for  the  pile  placement  data,  the  1-rater  and  n-rater  reliabili¬ 
ties  did  improve  for  the  three  decks  (B,  D,  and  E)  for  which  one  or  more 
judges  had  been  removed.  The  1-rater  reliabilities  improved  from  about  .58 
to  .73  on  the  average.  In  contrast,  the  imputation  of  pile  placement  values 
for  the  combinations  that  were  not  missing  one  or  more  judges  had  prac¬ 
tically  no  effect  on  the  obtained  reliabilities. 

The  1-rater  and  n-rater  reliabilities  of  the  bU  common  MOS/performance 
level  combinations  were  also  obtained  for  both  the  nonedited  and  edited 
ratio  scale  data.  The  1-rater  reliability  rose  from  .564  for  the  nonedited 
data  to  .653  for  the  edited  data  and  the  ji-rater  reliability  (based  on  74 
judges)  increased  from  .990  to  .992.  When  the  pile  placement  reliabilities 
for  the  60  corrmor.  combinations  were  computed  for  comparison  purposes,  the  1- 
rater  reliability  rose  from  .673  to  .746,  while  the  n-rater  reliability  rose 


^Here,  the  correlations  are  computed  over  the  MOS/performance  level 
combinations  rated  in  common  by  the  judges.  The  performance  percentiles 
assigned  to  the  combinations  were  used  as  an  additional  independent 
variable. 
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Table  4.11 

Xntraclass  Reliabilities  for  Pile  Placement  Data  by  Deck 
(Coaaton  and  Noncoaaon  Combinations) 


Deck 

No.  of 
Raters 

1-Rater 

jl-Rater 

No.  of 
Raters 

1-Rater  xk“Rater 

A 

12 

.778 

.977 

12 

.774 

.976 

B 

13 

.548 

.939 

11 

.707 

.964 

C 

12 

.718 

.968 

12 

.717 

.968 

D 

13 

.622  . 

.954 

11 

.752 

.971 

E 

12 

.562 

.937 

9 

.733 

.961 

F 

12 

.657 

.958 

12 

.658 

.958 

Average 

.648 

.956 

.724 

.966 

from  .993  to  .995.  The  high  values  for  these  £-rater  reliabilities  indicate 
that  not  much  is  to  be  gained  by  such  editing  wh*-''  the  number  of  judges  is 
large. 

The  correlations  between  the  utilities  obtained  for  the  nonedited  and 
edited  data  were  very  high.  Table  4.12  shows  that  even  for  the  decks  (B,  D, 
and  E)  where  judges  were  dropped  because  they  were  not  typical,  the  correla¬ 
tions  of  mean  pile  placement  across  the  280  combinations  averaged  .990.  The 
corresponding  correlation  for  the  mean  direct  judgment  utilities  assigned 
the  60  conmon  combinations  using  nonedited  and  edited  data  was  .999.  While 
it  is  apparent  from  these  results  that  not  much  was  gained  by  editing  the 
data,  it  is  also  evident  that  the  editing  did  not  unduly  affect  the  resul¬ 
tant  relative  utility  values. 


Scaling  Method  Reliabilities.  The  small  differences  in  reliability 
obtained  for  the  edited  vs.  nonedited  data  should  not  obscure  the  finding 
that  the  average  scale  values  assigned  to  the  HOS/performance  combinations 
had  a  very  high  degree  of  reliability.  The  n-rater  reliabilities  for  the 
six  separate  decks  ranged  from  .958  to  .976  for  the  edited  pile  placement 
data.  The  n-rater  (67  judges)  reliability  for  the  edited  direct  judgment 
utilities  of  the  common  combinations  was  .992.  The  corresponding  reliabil¬ 
ity  for  the  pile  placements  of  the  common  combinations  (across  all  decks  anc 
the  67  judges)  was  .995.  The  correlation  obtained  between  the  average  scale 
values  from  the  two  methods  across  the  60  common  combinations  was  .98. 


This  high  correlation  is  net  wholly  attributable  to  judaes  simply 
assigning  higher  values  to  combinations  with  higher  percentiles.  This  can 
be  seen  by  the  correlations  between  average  pile  placement  and  direct 
judgment  utilities  attained  when  the  correlations  are  computed  across  the  12 
conmon  MCS  holding  percentile  level  constant.  These  correlations,  presented 
in  Table  4.13,  had  an  average  value  of  .77.  The  .n-rater  (67  judges) 
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Table  4.12 

Correlation  Batwaon  Nairn  Pile  Placonent  Using  Hcnedited  and  Editad  Oats 


Deck 

Humber  of 
Combination! 

Edited  vs. 
Honed i ted 
Correlation 

A 

280 

.9998 

B 

280 

.9930 

Ca 

270 

.9999 

D 

280 

.9929 

Eb 

275 

.9856 

F 

280 

.9998 

«  MOS  16L  and  27Q  were  rescinded  as  of  31  October  1987,  so  their  data 
were  deleted. 

b  MOS  24W  was  rescinded  as  of  31  October  1986,  therefore  its  data 
were  deleted. 


Table  4.13 


Correlations  and  Reliabilities  of  Pile  Placement  and  Direct  Judgment 
Scale  Utilities  Holding  Percentile  Level  Constant 


Percentile 

Number  of  Common 
Combinations 

Correlation  of 
Mean  PP  with  DJa 

n-Rater  Reliability 
Pile  Direct 

Placement  Judgment 

10 

12 

.85 

.89 

.67 

30 

12 

.84 

.90 

.68 

50 

12 

.59 

.88 

.83 

70 

12 

.63 

.82 

.95 

90 

12 

.95 

.94 

.97 

Average 

.77 

.89 

.82 

a  PP  ■  Pile 

Placement;  DJ  ■  Direct  Judgment 

reliabilities  averaged  .89  and  .82  respectively  for  the  pile  placement  and 
direct  judgment  utilities  when  the  reliabilities  were  computed  for  each 
percentile  level  separately. 

These  results  clearly  demonstrate  that,  in  making  their  utility 
judgments,  the  judges  were  reacting  to  more  than  the  percentile  levels 
assigned  to  the  combinations.  Figure  4.4  shows  the  bivariate  plot  of  mean 
pile  placement  and  mean  direct  judgment  by  percentile  level  for  the  60 
common  combinations. 
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MEAN  PILE  PLk_£MCMT 

Figure  4.4.  Plot  of  »ean  pile  p lac scant  and  direct  judgment  utilities  by 
Performance  Percentile  for  6C  ccasaon  combinations:  67  judges 


Illustrative  Interval  Scale  Utilities.  It  Is  of  Interest  to  note  which 
MOS/performance  level  combinations  received  the  lowest  and  highest  utili¬ 
ties.  Does  the  pattern  sees  reasonable?  Do  the  relative  values  sake  sense 
given  what  we  know  about  the  MOS?  Table  4.14  presents  the  noncoamon 
combinations  that  received  mean  pile  placements  of  1.1  or  less  bv  the  9  to 
12  officers  who  Judged  them.  (The  lowest  or  negative  utility  pile  was 
as'*gned  a  value  of  1.0.)  Poor  perforaance  (10th  percentile;  was  assigned 
th»  jost  negative  utility  for  repairers  of  complex  Military  equipment  and 
for  the  Cardiac  Specialist  and  Pharmacy  Specialist.  In  each  of  these  MOS, 
the  consequences  of  poor  perforaance  are  Judged  to  be  especially  costly. 


Table  4.14 

List  of  MOS/Perforaance  Level  Coablnatlons  Receiving  Lowest  Pile  Placements 
(Mean  Placement  1.1  and  Below) 


MOS  IP* 

rnsnm 

23H 

NIKE/HERCULES  Track  Radar  Repairer 

34Y 

Field  Artillery  Tactical  Fire  Direction  Systems  Repairer 

35R 

Avionic  Special  Equipment  Repairer 

63N 

M60A1/A3  Tank  System  Mechanic 

67N 

Utility  Helicopter  Repairer 

67R 

AH-64  Attack  Helicopter  Repairer 

67Y 

AH-1  Attack  Helicopter  Repairer 

688 

Aircraft  Powerplant  Repairer 

68F 

Aircraft  Electrician 

91N 

Cardiac  Specialist 

91Q 

Pharmacy  Specialist 

aA11  associated  performance  levels  were  at  the  10th  percentile. 


Table  4.15  lists  the  noncomroon  combinations  that  received  the  highest 
mean  pile  placements  (means  of  6.9  or  more  on  a  scale  of  7).  This  list 
Includes  both  repairers  and  operators  of  advanced  weapon  systems.  High 
performance  in  two  Intelligence  MOS,  968  and  98C,  was  also  assigned  a  high 
utility.  In  comparison,  the  mean  pile  placement  for  the  90th  percentile 
Infantryman  (118)  was  6.6. 
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Copparison  of  Utility  Ratings  by  Different  Officer  Groups.  To  further 
test  whether  the  type  of  judge  influenced  utility  ratings,  a  separate  linear 
regression  equation  was  computed  for  each  of  the  12  common  HOS  for  each 
judge,  using  performance  percentiles  as  the  independent  variable  and  the 
direct  judgment  utilities  assigned  by  the  officer  as  the  dependent  variable. 
The  taean  slopes  and  the  mean  y-lntercepts  or  equation  constants  of  the 
officer  groups  were  then  analyzed  for  significant  differences,  using 
separate  analysis  of  variance  for  the  12  regression  slope  Deans  and  the  12 
Intercept  neans. 

Analyses  were  conducted  to  determine  whether  officers  in  different 
primary  military  specialties  assigned  significantly  different  values  to  the 
various  combinations.  Similar  sets  of  analyses  of  variance  tests  were  run 
comparing  the  utilities  assigned  by  Combat,  Combat  Support,  and  Combat 
Service  Support  officers.  In  addition,  the  utilities  assigned  by  the  47 
majors  in  the  sample  were  compared  to  those  assigned  by  a  combined  group  of 
20  lieutenant  colonels  and  colonels. 

In  all,  only  10  of  the  more  than  2S0  statistical  tests  run  to  compare 
various  types  of  officer  groups  were  significant  at  the  .05  level.  Examina¬ 
tion  of  the  significant  differences  that  were  obtained  did  not  reveal  any 
trend  in  the  data  indicating  that  certain  types  of  officers  favored  particu¬ 
lar  MOS  or  performance  levels. 

Estimation  of  Ratio  Sctlt  Utilities  From  Pile  Placement  (Interval) 

Data.  A  basic  objective  of  the  overall  research  design  was  to  place  all 
1.380  MOS/ performance  level  combinations  on  the  same  utility  scale.  Using 
the  averages  (across  all  judges)  of  the  direct  judgment  utilities  assigned 
the  60  cofimon  combinations  as  the  dependent  variable,  and  the  pile  placement 
of  the  same  common  combinations  as  the  basic  independent  variable,  an 
equation  was  derived  for  each  separate  group  of  judges  expressing  direct 
judgment  utilities  as  a  function  of  average  pile  placement. 

This  equation  was  then  used  to  estimate  the  ratio  scale  values  (direct 
judgment  utilities)  that  each  group  would  have  assigned  all  the  noncommon 
MOS/performance  level  combinations  if  they  had  been  given  that  scaling  task. 
It  was  assumed  that  since  these  equations  would  place  all  the  estimated 
utility  values  on  the  same  scale,  minor  group  differences  in  pile  placements 
of  the  60  coirafiori  combinations  would  be  averaged  out.  (As  the  judges  were 
assigned  randomly  to  decks,  any  differences  among  the  groups  in  mean  pile 
placements  could  be  attributed  to  sampling  error. ^) 

To  explore  the  use  of  alternative  regression  equations  for  estimating 
the  ratio  scale  values  from  the  pile  placement  data,  a  subset  of  20  of  the 
common  combinations  was  temporarily  set  aside  and  not  used  in  the  initial 
derivation  of  the  reregression  equations.  These  20  combinations  came  from 


^Analysis  of  variance  significance  tests  run  on  the  59  common 
combinations  to  compare  their  mean  pile  placements  by  deck  resulted  in  only 
one  significant  difference,  a  result  easily  attributable  to  chance. 
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four  MOS  having  performance  percentiles  that  had  utilities  fairly  evenly 
spread  across  the  range  of  ratio  scale  utilities  (based  on  the  67- judge 
averages). 

Since  both  estimated  and  actual  values  would  be  available  for  these  20 
combinations,  the  ability  of  alternative  regression  equations  to  estimate 
ratio  scale  utilities  from  the  pile  placement  data  could  be  evaluated  or 
cross  validated.  That  is,  a  regression  equation  from  each  deck  based  on  40 
common  combinations  could  be  used  to  estimate  the  ratio  scale  values  of 
the  20  set-aside  combinations,  for  which  actual  values  were  available. 

Two  indexes  of  how  well  the  estimated  ratio  scale  values  corresponded 
to  the  actual  values  were  (a)  the  mean  difference  between  the  actual  and  the 
estimated  values,  and  (b)  the  square  root  of  the  mean  square  of  the  differ¬ 
ence  between  the  actual  and  estimated  values.  The  two  indexes  were  computed 
using  five  different  sets  of  Independent  variables: 

(1)  Average  pile  placement  of  the  40.  combinations. 

(2)  Average  pile  placement  and  the  performance  percentiles 
of  the  40  combinations. 

(3)  Average  pile  placement  and  the  square  of  the  average  pile 
placement. 

(4)  Average  pile  placement  and  the  cube  of  the  average  pile 
placement. 

(5)  Average  pile  placement  and  both  the  square  and  cube  of 
average  pile  placement. 

The  square  and  the  cube  of  the  average  pile  placements  were  used  as 
independent  variables  because  the  by-deck  bivariate  plots  of  the  average 
ratio  scales  (the  dependent  variable)  versus  the  average  pile  placement  (the 
Independent  variable)  suggested  that  there  might  be  inflections  in  the  best 
fitting  lines  at  the  two  ends  of  the  utility  continuum.  This  might  be 
brought  about  by  the  restriction  inherent  in  the  procedure  used  that  limited 
the  judges  to  seven  utility  levels  when  placing  the  combinations  into  piles. 

Table  4.16  shows  the  results  of  these  analyses.  The  equations  based  on 
all  five  sets  of  independent  variables  tended  to  overestimate  somewhat  the 
actual  utilities  of  the  20  holdout  combinations.  This  tendency  was  most 
pronounced  for  combinations  having  intermediate  actual  utilities.  In 
general,  the  equations  tended  to  underestimate  the  utilities  of  the  holdout 
combinations  having  high  actual  utilities,  and  slightly  overestimate  the 
utilities  of  the  combinations  having  low  actual  utilities.  The  best  balance 
was  achieved  by  using  average  pile  placement  and  both  its  square  and  its 
cube  as  the  independent  variables. 

The  lowest  mean  squares  for  prediction  errors  was  also  obtained  by  the 
equations  that  used  the  average  pile  placement  and  both  the  square  and  the 
cube  of  average  pile  placement.  For  all  equations,  the  largest  squared 
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Tabl®  4.16 

Comparison  of  Equations  Estimating  Ratio  Scale  Utilities  From 
PIT®  Placement  Data  (n  •  20  Combinations  In  Each  of  6  Decks) 


Eauation  Indeoendent  Variable 

Average 

iL^JL la 

Sq.  Root 
of 

Average,, 

Li  -  ii2 

Average  R^ 

Average  pile  placement  (PP) 

-  .87 

9.15 

.943 

PP  and  performance  percentile 

PP  and  (PP}2 

-1.65 

10.01 

.950 

-  .86 

9.04 

.944 

PP  and  (PP)3 

-  .86 

9.01 

.944 

PP.  (PP)2,  and  (PP)3 

-  .79 

8.96 

.945 

•Actual  ratio  scale  utility  (A)  minus  estimated  ratio  scale  utility  (E). 
“Forty  cownon  combinations  were  used  In  each  deck  to  obtain  the  squares  of 
the  multiple  correlation  coefficients  (Br). 


errors  were  for  the  combinations  having  actual  abilities  In  the  mid-range. 
As  mentioned  above,  the  equations  slightly,  but  consistently,  overestimated 
these  utilities. 


Based  on  these  data,  pile  placement  and  its  square  and  cube  were  used 
as  the  independent  variables.  The  equations  tor  each  deck  were  recomputed, 
using  the  pile  placement  and  direct  judgment  values  for  all  60  combinations. 
Table  4.17  presents  the  adjusted  correlation  coefficients  (Rz)  obtained  for 
each  deck,  as  ..-el 1  as  the  actual  equation  weights.  The  multiple  correlation 
coefficients  remained  high  (about  .97  on  the  average).  The  sign  of  the 
weights  obtained  formed  a  fairly  consistent  pattern,  with  average  pile 
placement  always  having  a  positive  weight,  and  (except  for  Deck  A)  the 
square  and  the  cube  of  average  pile  placement  having  negative  and  positive 
weights,  respectively. 


Cross-Vaiidation  of  Estimation  Equations  on  a  Holdout  S«mp1®.  How  would 

sticp.5  given  in  Teble  4,17  cofnD^re 


*ne  uT_  iii  vy 


ICIUCi 


I  I  UI4I 


WUV 


with  utilities  obtained  from  direct  judgments  by  field  grade  officers?  To 
explore  this  question,  the  participants5  in  the  last  utility  workshop  were 
given  an  additional  40  combinations  (8  MOS  x  5  levels)  on  which  to  make  their 


5None  of  the  10  participants  were  Armor  officers,  unlike  the  remaining 
sample  where  17  of  the  58  officers  were  Armor  officers.  One  of  the  10  was 
Judge  ID  #83  who,  as  noted  earlier,  was  removed  from  the  sample  because  of  a 
large  number  of  Inversions  and  low  medium  correlations  with  the  other  judges. 
Hence  the  final  analyses  from  this  workshop  were  based  on  nine  participants. 
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Table  4.17 


Multiple  Regression  Equations  for  Estimating  Ratio  Scale  Utilities 
Froai  Average  Pile  Placement  Data  (Based  on  60  Comoh  Combinations) 


Independent 

Variable 

PI le.pl acenent  (PP) 

jppji 

Intercept ' 

Adjusted  fi2 


....  A  . ,  _ B _ 

14.00  21.81 

1.455  -.323 

-.053  .067 

-34.09  -41.75 

.965  .926 


_ Deck _ 

_ £ _ _ ,Q _ 

43.39  24.09 

-5.785  -1.344 

.485  .169 

-69.44  -47.74 

.954  .944 


_ l _  _ E _ 

46.99  49.45 

-6.922  -6.932 

.545  .549 

-63.80  -77.85 

.912  .924 


direct  judgments  of  utility.  The  mean  of  the  direct  judgment  utilities 
given  these  40  combinations  by  these  officers  could  then  be  compared  to 
values  from  the  other  58  officers  computed  by  formulas  derived  for  each  deck 
excluding  the  data  obtained  from  the  last  workshop. 

Before  the  utility  comparison,  the  set  of  40  new  utility  estimates  from 
the  holdout  sample  were  adjusted  so  that  the  new  values  corresponded  to  the 
utilities  that  could  have  been  expected  if  the  remaining  sample  of  58  judges 
had  actually  evaluated  the  additional  40  combinations  using  the  direct 
judgment  procedure.  A  multiple  regression  equation  was  derived  using  the 
mean  direct  judgment  utility  assigned  the  60  common  combinations  by  the  58 
officers  as  the  dependent  variable,  and  the  mean  utilities  assigned  the  60 
combinations  by  the  nine  officers,  the  square  of  these  utilities,  and  the 
performance  percentile  of  the  60  combinations  as  the  Independent  variables. 
This  equation  was  then  used  to  obtain  the  estimated  direct  judgment  values 
for  the  40  extra  combinations.  This  procedure  adjusted  the  holdout  sample 
utilities  for  random  and  nonrandom  differences  between  the  holdout  and 
remaining  sample  judges  in  the  direct  judgment  of  the  60  common 
combinations. 

As  shown  in  Table  4.18,  very  high  correlations  (.97)  were  obtained 
between  the  utilities  estimated  from  the  separate  deck  equations  and  the 
holdout  sample  unadjusted  and  adjusted  direct  judgment  utilities.  Moreover, 
the  utility  means,  standard  deviations,  and  ranges  for  the  40  extra  combina¬ 
tions  obtained  from  the  holdout  sample  direct  judgments  were  quite  similar 
to  those  estimated  from  the  deck  equations.  This  was  especially  evident 
aft  t  adjustment  for  utility  differences  between  the  two  officer  samples  on 
the  r  direct  judgment  of  the  60  cotrrixjn  combinations. 
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Table  4.18 

Comparison  of  Utilities  Obtained  for  40  Extra  NOS/Perforaance 
Laval  Combinations 


Origin  of  Variable 
Utllitv  Estimate 

Judaes  Mean 

Standard 

Deviation 

Ranae 

(1) 

Oirect  judgments  by 
holdout  sample  of 
officers 

9  62.8 

38.4 

-9.7  to  123.7 

(2) 

Variable  1  adjusted  for 
differences  between  the 
holdout  and  remaining 
samples  on  GO  common 
combinations 

9,58  54.2 

36.5 

-5.8  to  110.8 

(3) 

Separate  deck  equations 
based  on  data  from 
remaining  officer  sample 
only 

58  56.3 

36.1 

-8.2  to  111.7 

Intercorrelations 

(1) 

(2) 

2)  .993a 

(3)  .972 

•  m  «• 

.973 

Variable  1  was  used  In  the  derivation  of  Variable  2. 


THE  FINAL  UTILITY  VALUES 

All  the  analyses  that  have  oeen  described  were  performed  for  the 
purpose  of  establishing  the  reliability  and  validity  of  the  utility  esti¬ 
mates  for  MOS  by  Performance  Level  combinations  obtained  by  using  the 
technique  described  ir.  this  chapter.  All  of  these  reliability  and  validity 
analyses  were  based  on  data  that  had  been  carefully  edited  for  missing  data 
and  outliers.  The  analyses  support  the  conclusions  that: 

(1)  For  both  methods  the  reliability  of  a  single  judge  is  reasonably 
high. 

(2)  For  both  methods  the  reliability  of  the  average  value  produced  by 
11  judges  or  more  is  very  high. 
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(3)  Reliabilities  are  high  even  when  performance  level  Is  controlled 
and  differences  are  due  only  to  MOS  differences  within  performance 
level. 

(4)  The  agreement  between  the  two  methods  when  scale  values  are 
compared  on  a  set  of  common  stimuli  is  very  high  and  equal  to  the 
limit  of  their  reliabilities. 

(5)  Officers  of  different  ranks  or  HQS  specialties  do  not  produce 
different  patterns  of  scale  values. 

(6)  Patterns  of  high  and  low  utility  values  "make  sense." 

(7)  A  relatively  simple  exercise  in  equation  fitting  produced  a  useful 
method  for  estimating  ratio  scale  values  (which  could  not  be 
obtained  for  all  MOS  by  Performance  Level  combinations)  from  the 
interval  scale  values  obtained  from  all  MOS  by  Performance  Level 
combinations,  using  the  pile  placement  method. 

(8)  As  determined  on  a  cross-validation  sample  of  stimuli,  the  equa¬ 
tions  used  to  estimate  ratio  values  from  interval  data  were  highly 
accurate  (Restimated  x  ac*u4'  -.97). 

Utilities  for  HQS /Performance  Level  Combinations 


The  derived  equations  for  each  deck  were  used  to  estimate  the  ratio 
scale  utilities  for  the  noncommon  MOS/performance  level  combinations  for  the 
entry-level  list  of  273  HG5.  These  values  are  given  in  Appendix  F,  along 
with  the  actual  average  direct  judgment  utilities  for  the  60  common  combi¬ 
nations. 


Consequently,  Appendix  F  represents  the  “bottom  line"  of  the  Project  A 
utility  scaling  work  at  the  end  of  F87.  It  contains  ratio  scale  utility 
values  for  273  x  5,  or  1,365,  MOS/performance  level  combinations.  Within 
the  limits  of  the  reliability  and  validity  evidence  discussed  in  this 
report,  the  1,365  combinations  have  been  placed  on  the  same  scale.  As  an 
example  of  this  extended  MOS  list  of  utility  values,  the  ratio  scale  values 
for  the  19  Project  A  MOS  are  shown  In  Table  4.19. 

Equations  for  Estimating  Utilities  for  Continuous  Performance  Distributions 

To  make  it  possible  to  assign  a  utility  value  to  any  performance 
percentile  within  an  MOS  (not  just  the  10,  30,  50,  70,  and  90  percentiles), 
a  separate  equation  was  derived  for  each  of  the  273  MOS  relating  performance 
percentile  and  the  square  and  cube  of  performance  percentile  (the  indepen¬ 
dent  variables)  to  utility  (the  dependent  variable;. 

Each  equation  was  based  on  five  data  points,  the  estimated  (or  actual) 
average  ratio  scale  utilities  respectively  assigned  the  10th,  30th,  50th, 
70th,  and  90th  percentile  levels  within  the  MOS.  To  determine  the  general 
shape  of  the  relationship  between  percentile  level  and  utility  across  the 
MOS,  a  stepwise  multiple  regression  procedure  (SAS)  was  used.  The  order  of 
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Tabic  4.19 


Average  Ratio  Seal#  Utilities  by  Performance  Percentile 
For  the  Project  A  NOS 


Percentile 


*05 

10 

30 

50 

70 

50 

11B* 

4.5 

38.7 

61,7 

82.3 

100.0 

12B 

5.1 

47.2 

63.3 

86.8 

112.6 

13B 

20.8 

35.0 

67.1 

85.2 

111.7 

16S 

-8.2 

33.4 

40.3 

77.3 

105.6 

19E 

8.4 

45.6 

68.3 

98.0 

108.2 

27E 

-13.8 

31.0 

50.7 

91.1 

115.2 

31C 

-6.4 

25.4 

54.3 

86.8 

97.9 

51B 

22.7 

36.9 

54.0 

65.9 

59.3 

54E 

1.8 

31.7 

45.4 

89.1 

108.7 

55B 

0.6 

40.1 

61.5 

82.6 

100.2 

63B 

0.5 

35.0 

55.1 

87.7 

111.7 

64C 

22.7 

48.8 

52.2 

81.9 

79.6 

67N 

-24.0 

8.5 

46.9 

80.5 

107.7 

71L 

0.5 

27.3 

54.3 

70.8 

86.8 

76W 

15.5 

39.2 

59.1 

73.5 

82.7 

76Y 

-5.0 

33.1 

66.4 

81.5 

92.7 

91A 

-4.0 

17.6 

52.5 

76.6 

100.2 

94B 

2.7 

27.4 

63.9 

85.5 

90.8 

95B 

-8.2 

38.6 

63.1 

84.2 

108.7 

*0r.e  of  12  common  MOS  assigned  actual  ratio  scales. 


entry  of  the  independent  variables  into  the  equation  and  the  significant 
levels  and  signs  (positive  or  negative)  of  their  regression  weights  were 
noted  for  each  HOS  equation. 

The  equations  derived  for  the  273  MOS  are  summarized  in  Table  4.20. 
Performance  percentile  was  the  first  independent  variable  to  enter  into  the 
equation  for  each  MOS.  For  91  of  the  MOS  equations,  a  second  independent 
variable  entered  the  equation  with  a  statistically  significant  weight  (.10 
level  of  significance).  For  the  most  part  (95V),  this  second  variable 
entered  with  a  negative  weight,  indicating  that  the  rate  of  increase  in 
utility  was  declining  for  higher  performance  percentiles.  The  cube  of 
performance  percentile  was  selected  as  the  second  equation  variable  a  little 
more  often  than  the  square  of  performance  percentile  (52  vs.  39  times).  For 
only  three  of  the  91  equations  did  a  third  independent  variable  also  enter 
the  equation  significantly  (.10  level).  This  would  indicate  a  best  fitting 
line  with  two  inflection  points. 
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Table  20 


Sumnary  of  HOS-Specific  Equations  Ocrived  for  Estimating  the  Utility  of 

Performance  at  Different  Percentile  Levels 

Indeoendent  Variable 

First  Variable 
Entering 
Significantly 
Into  Eauation 

Sign  of 

Regression  Weight 
♦  - 

Second  Variable 
Also  Entering 
Significantly 
Into  Equation 

Sign  of 

Regression  Weight 
♦  ■» 

Third  Variabl* 
Also  Entering 
Significantly 
Into  Eauation 

Sign  of 

Regression  Weigt 
♦  - 

Performance  percentile  (P) 
Percentile  squared  (P2) 

273  0 

0  0 

0  0 

0  0 

1  38 

!  1 

Percentile  cubed  (P^) 

JL 

_A_  -4S. 

_L.  JL 

Total 

273  0 

S  86 

2  1 

Figure  4.5  shows  bivariate  plots  between  utility  and  performance 
percentile  within  three  MOS.  Line  B  in  the  figure  shows  a  typical  plot  when 
the  relationships  are  essentially  linear  between  utility  and  performance. 

An  example  of  a  plot  where  a  Second  variable  (perCentile  Squared)  enters 
into  the  regression  equation  with  a  negative  weight  is  provided  by  line  A 
while  line  C  shows  an  example  where  the  second  independent  variable  entering 
the  equation  is  percentile  cubed. 

Notice  the  very  high  values  of  adjusted  fi2  in  the  examples  shown.  The 
five  data  points  in  the  plots  were  determined  from  utility  judgment  data. 

As  shown  earlier  (see  Table  4.13),  reliable  differences  between  the  util¬ 
ities  assigned  MOS  are  obtained  at  all  percentile  levels  when  performance 
percentile  is  held  constant.  Within  MOS,  however,  utility  is  highly 
predictable  from  performance  percentile,  though  the  relationship  is  fre¬ 
quently  not  linear  over  the  range  of  performance. 

The  operational  significance  of  these  findings  is  that  the  utility  of 
assigning  a  recruit  to  any  MOS  can  be  estimated  using  a  within-MOS  equation 
to  relate  the  level  of  the  recruit's  predicted  performance  in  the  MOS  to  his 
or  her  utility  for  that  MOS.  These  utilities,  in  turn,  could  be  used  to 
help  decide  the  MOS  to  which  the  recruit  should  be  assigned  under  an 
algorithm  for  optimizing  job  assignments. 
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DISCUSSION  AND  CONCLUSIONS 


Perhaps  the  most  Important  finding  In  this  first  research  effort  to 
establish  utility  values  for  different  levels  of  performance  In  all  Army 
entry-level  HOS  is  that  it  can  be  done.  One  can  fault  the  subjective  nature 
of  the  Judgments  called  for,  but  the  fact  remains  that  the  mean  utilities 
had  very  high  reliabilities,  both  within  and  between  methods  (for  example, 
for  the  coanon  set  of  combinations,  .99  within  the  utility  scaling  method 
and  .98  between  methods).  The  high  correlation  (.97)  and  similarity  between 
the  utilities  derived  from  separate  deck  equations  and  the  utilities 
assigned  directly  by  the  holdout  officer  sample  (Table  4.18)  further  attests 
to  the  stability  of  the  utilities  across  methods  and  officer  samples. 

The  high  reliabilities  of  the  mean  utilities  assigned  the  MOS/perfor- 
mance  level  combinations,  and  the  lack  of  any  clear  pattern  of  differences 
in  average  utilities  assigned  by  officers  from  different  HOS  specialties, 
also  indicate  that  similar  values  would  result  if  the  utilities  were 
assigned  by  a  different  sample  of  officers  than  the  one  used  in  this 
research.  Field  grade  Army  officers  apparently  share  similar  perceptions  of 
the  relative  worth/costs  of  low  and  high  performance  in  Army  MOS.  This 
research  clearly  demonstrates  that  this  shared  organizational  value  function 
can  be  reliably  scaled. 

The  finding  that  consistent  differences  in  the  utility  of  performance 
are  obtained  for  different  MOS  at  eacb  percentile  level  is  also  worth 
noting,  especially  when  combined  with  the  finding  that  the  relative  ranking 
of  the  MOS  in  terms  of  their  utility  levels  shifts  depending  upon  the 
predicted  performance  level.  A  personnel  assignsitnl  algorithm  that  took 
these  utility  differentials  into  account  at  all  performance  levels  would 
most  likely  be  able  to  effect  more  optimal  Army-wide  assignments  than  one 
that  did  not. 

However,  a  number  of  critical  problems  need  to  be  addressed  before 
utilities  similar  to  the  ones  obtained  in  t’ns  research  can  be  used  opera¬ 
tionally.  Foremost  perhaps  is  the  problem  of  how  to  ensure  that  the  proper 
distribution  of  available  personnel  talent  'S  assigned  to  each  MOS.  An 
assignment  algorithm  that  paid  attention  only  to  the  utility  of  assigning 
individual  recruits  to  MOS  or  CMF,  without  regard  to  the  utility  of  the 
total  distribution  of  low-  or  high-quality  personnel  being  assigned  to  each 
job,  could  result  in  certain  jobs  being  filled  by  insufficient  numbers  of 
technically  proficient  recruits.  Research  to  determine  the  utility  of 
different  distributions  of  available  recruits  in  Army  jobs  is  the  subject  of 
an  ongoing  parallel  effort  being  undertaken  by  ARI  researchers  (Nord  & 

White.  1987). 

Another  issue  concerns  obtaining  the  acceptance  of  these  utility  values 
by  those  who  are  responsible  for  personnel  policies  and  decisions.  Such 
approval  is  unlikely  to  occur  unless  it  can  be  demonstrated  that  use  of  the 
utility  information  would  result  in  more  optimal  manpower  allocations.  Work 
is  now  in  progress  (Nord  4  White,  1987)  to  examine  the  effects  of  using  (or 
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not  using)  these  job-specific  utility  functions  to  sake  personnel  classi¬ 
fication  aRd  job  assignment  decisions. 

Yet  another  issue  concerns  the  duration  of  tine  that  the  recruits 
actually  iair.  in  the  Army.  All  other  things  being  equal,  recruits  who 
complete  their  first  tour  of  enlirtaent  will  have  higher  utility  than  those 
who  do  not  Similarly,  high-quality  soldiers  who  reenlist  will  have  more 
utility  to  the  Army  than  those  who  fail  to  reenlist. 

A  related  consideration  is  cost.  Recruiting,  training,  maintaining, 
and  retaining  high-quality  soldiers  is  a  costly  operation.  These  costs  are 
not  equal  across  A ray  jobs.  It  costs  more  to  recruit  high-quality  personnel 
than  it  does  to  obtain  recruits  of  lower  quality.  Likewise,  it  nay  take 
longer  and  be  more  costly  to  train  soldiers  in  high-technology  career 
management  fields  than  in  some  other  types  of  CMF.  Obviously,  potential 
cost,  reenlistment  propensity,  and  NCO  potential  all  should  be  considered, 
if  at  all  possible,  in  making  the  Initial  assignments  of  recruits  to  jobs. 

Finally,  assuming  that  judged  utility  of  performance  has  a  role  in  an 
optimal  classification  and  job  assignment  system,  questions  remain  concern¬ 
ing  how  the  requisite  judgments  should  be  obtained  operationally.  What 
types  of  officers  should  make  the  judgments  Involved?  How  often  do  the 
resultant  utility  functions  have  to  be  updated  to  keep  the  utilities 
current?  Do  the  utilities  of  all  entry-level  MGS  have  to  be  determined,  or 
can  the  utilities  for  most  M0$  be  inferred  from  those  assigned  a  representa¬ 
tive  sample  of  MOS  taken  from  career  management  fields  or  other  MOS 
groupings? 

Clearly,  this  research  has  affirmatively  answered  tne  question  of 
whether  a  coherent,  reliable  set  of  relative  utility  values  could  be  derived 
for  all  performance  levels  in  all  entry-level  Army  MOS.  The  next  steps 
involve  how  to  mcke  best  use  of  that  finding  in  improving  the  Army's 
selection,  classification,  and  assignment  processes. 
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Chapter  5 

JOB  ANALYSIS  OF  SECCND-TOUR  PERFORMANCE1 
INTRODUCTION 


As  Project  A  entered  its  fifth  year,  a  major  added  area  of  criterion 
development  centered  on  job  analyses  and  performance  assessment  for  Army 
enlisted  personnel  who  have  been  in  the  Service  for  3  to  5  years  —  a  cohort 
that  we  now  refer  to  as  the  'second  tour.'  The  overall  goals  of  this  later 
part  of  the  project  include  developing  selection/classification  procedures 
that  will  aid  in  identifying  accessions  who  have  high  potential  to  become 
successful  noncommissioned  officers  (N£0s)  after  reenlistment. 


General  Approach 

To  encompass  NCO  potential  in  the  Project  A  research  domain,  the 
original  Research  Plan  provides  for  a  second-tour  followup  of  soldiers 
tested  in  both  the  83/84  Concurrent  Validation  sample  and  the  86/87 
Longitudinal  Validation  sample.  As  described  in  earlier  reports  (e.g., 
Campbell,  1986a),  the  CV  sample  was  made  up  of  individuals  who  entered  the 
Army  in  the  83/84  "window*  and  were  assessed  on  the  predictor  battery  and 
criterion  measures  during  1935/86.  The  LV  sample  was  tested  during  1986/87 
on  the  experimental  predictor  battery  as  they  entered  the  Array  and  will  be 
assessed  in  1988  on  the  performance  measures.  The  second-tour  followup  of 
the  83/84  cohort  will  involve  extensive  measurement  of  the  performance  of 
the  individuals  who  were  assessed  during  the  CV  in  1985  and  are  still  in  the 
Army  in  1988.  The  second-tour  followup  of  the  86/87  cohort  will  be 
conducted  in  1991. 

To  identify  valid  selection/classification  procedures  for  assessing  NCO 
potential,  measures  of  second-tour  job  performance  are  needed.  After  the 
criteria  are  available,  the  following  questions  can  be  examined: 

•  To  what  extent  does  the  experimental  predictor  battery 
predict  performance  beyond  the  first  term  of  enlistment? 

•  Does  early  performance  predict  later  performance,  when 
additional  responsibilities,  such  as  supervision  and 
leadership,  are  presumably  required? 

•  What  combination  of  selection/classification  test  information 
and  first-tour  performance  data  is  most  effective  for  predicting 
second-tour  performance? 


*The  initial  draft  of  this  chapter  was  written  by  Charlotte  H. 
Campbell  of  the  Human  Resources  Research  Organization. 
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•  How  does  entry-level  training  performance  relate  to  later 
Job  performance  in  the  first  and  second  tours? 

Before  criterion  measures  are  developed  and  the  above  questions  addressed, 
"second-tour*  job  analyses  of  the  NCO  position  needed  to  be  made. 

In  the  first-tour  job  analyses  In  earlier  Project  A  research,  we  used 
both  task-based  and  critical  incident-based  methods  to  obtain  the  needed 
information.  The  task-based  method  involved  heavy  reliance  on  existing  job 
information,  supplemented  by  interviews  with  job  experts,  to  first  identify 
all  the  relevant  tasks  encompassed  by  a  job;  from  this  list,  a  smaller  set 
of  tasks  was  chosen  to  represent  the  full  domain  for  measurement  purposes. 
The  critical  incident-based  method  Involved  conducting  a  series  of  workshops 
in  which  job  experts  generated  examples  of  good,  poor,  and  average 
performance.  These  examples  were  then  clustered  into  dimensions  of 
performance.  In  a  fashion  similar  to  that  used  in  developing  behavioral ly 
anchored  rating  scales  (BARS)  (Toquam,  et  al.,  1986). 

This  information-gathering  approach  was,  In  our  judgment,  reasonably 
successful  for  the  earlier  analysis  of  entry-level  jobs.  However,  second- 
tour  soldiers  were  expected  to  have  supervisory  and  leadership  responsibil¬ 
ities  as  well  as  technical  job  requirements.  Supervisory  behaviors  tend  to 
be  continuous  rather  than  discrete,  are  not  easy  to  observe  and  measure,  and 
are  difficult  to  fix  in  time  (Rumsey,  1987).  Consequently,  the  basic  job- 
analytic  methods  used  earlier  were  modified  and  extended,  as  described  in 
this  chapter,  to  acconroodate  these  expected  properties  of  the  second-tour 
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Specific  Objectives  for  the  $econd-Tour  Job  Analysis 

The  overall  aoal  of  the  second-tour  job-analytic  work  was  to  define  the 
domain  of  higher  level  performance  requirements  for  a  representative  sample 
of  Army  enlisted  jobs.  Specifically,  the  objectives  were  to: 

•  Describe  the  major  differences  between  entry-level  and  higher 
level  performance  content,  within  Military  Occupational 
Specialty  (MOS). 

•  Describe  the  major  differences  across  MOS  within  the  higher  level 
jobs. 

•  Describe  the  specific  nature  of  the  supervisory/leadership 
component  of  these  higher  level  jobs. 

Once  these  objectives  were  achieved,  and  full  descriptions  within  and 
across  MOS  were  available,  the  information  would  be  used  to  address  four 
questions  about  measurement  of  HCO  potential: 

•  What  should  be  the  conte  t  of  the  new  criterion  measures? 

•  What  kinds  of  measurement  methods  are  needed? 
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•  Are  separate  measures  needed  for  each  job?  Or  are  the  Jobs  so 
similar  that  the  same  measures  can  be  applied  to  all? 

•  To  what  extent  can  aeasures  developed  for  entry-level  soldiers  be 
used  among  higher  level  soldiers? 

The  methods  and  results  of  the  multiple  approaches  used  to  analyze  the 
higher  level  positions  are  described  In  this  chapter. 


The  Second-Tour  Cohort 

The  second-tour  (as  noted  earlier.  Project  A  uses  this  term  to  desig¬ 
nate  soldiers  who  have  been  In  the  Army  3  to  5  years)  samples  were  taken 
from  the  nine  MOS  (known  as  Batch  A)  that  had  been  originally  designated  for 
more  intensive  job-specific  performance  measurement  during  the  first-tour 
data  collections.  They  were  drawn  from  the  full  sample  of  19  Project  A  MOS 
that  were  selected  because  of  the  following  characteristics:  (a)  high- 
density  MOS  thdt  would  provide  sufficient  sample  size'  for  statistically 
reliable  estimates  of  new  predictor  validity  and  differential  validity 
across  racial  and  gender  groups;  (b)  representative  job  content;  (c)  repre- 
sentative  of  the  Army's  designated  Career  Management  Fields;  (d)  high- 
priority  MOS,  as  rated  by  the  Army  in  the  event  of  a  national  emergency;  (e) 
representative  of  combat,  combat  support,  and  combat  service  support  MOS. 

The  paygrade  of  these  "second-tour"  soldiers  at  the  time  of  data 
collection  will  vary  from  one  MOS  tc  another  because  of  differences  in  MOS 
density  and  Army  promotion  needs,  which  affect  reenl istment  rates  ana 
promotion  opportunities  (Table  5.1).  We  expect  that,  depending  on  MOS,  from 
15  to  68  percent  will  be  classed  as  Skill  Level  2  (E5  NCOs).  Most  of  the 
others  will  be  Skill  Level  1  (El  through  E4;  most  will  probably  be  E4),  and 
a  very  few  will  be  Skill  Level  3  (E6  NCOs). 

There  were  several  reasons  for  defining  the  second-tour  cohort  in  terms 
of  time  in  service  rather  than  in  terms  of  paygrade,  skill  level,  or 
reenlistment  rate.  None  of  these  designations  would  produce  a  "clean"  or 
"pure"  cohort;  not  everyone  with  3  to  t  years  of  service  will  be  at  the  same 
paygrade  or  skill  level,  and  reenlistment  rates  differ  across  MOS.  The 
situation  would  be  even  less  clear  if  the  cohort  were  defined  as  E5  or  as 

Skill  Level  2.  For  example,  at  any  given  time  many  £4s  are  filling  E5 

positions  and  there  is  no  reasonable  way  to  account  for  all  the  situational 
contingencies  that  lead  to  this  result;  thus,  the  distinctions  between  E4 
and  E5  may  not  correspond  to  the  actual  duty  positions  being  filled.  Also, 
"skill  level"  is  a  task  designation,  and  the  number  and  proportion  of  Skill 
Level  2  tasks  vary  widely  across  MOS.  At  any  given  time  a  particular  Skill 

Level  2  task  might  be  performed  by  an  E4,  F5,  or  E6. 

If  the  cohort  to  be  sampled  is  defined  by  time  since  Basic  Training, 
the  measurement  goal  is  to  predict  performance  at  a  future  date,  not  to  pre¬ 
dict  performance  on  Skill  Level  2  tasks,  or  as  an  E5,  whenever  that  occurs. 
Consequently,  the  rate  at  which  an  individual  has  progressed  will  become 
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Tibi*  5.1 


Projected  P*rc*nt*g*  of  S*cond-Tour  Cohort  by  Skill  L*v*1  it  Tim  of  Diti 
Collection  Planned  for  Summer,  1988 


Reenllstaent  Skill  Level 


MOS 

Percent 

.  SUL 

SL2 

SL3 

11B 

Infantryman 

27 

41 

58 

1 

13B 

Cannon  Crewman 

31 

70 

30 

0 

19E 

Armor  Crewman 

38 

31 

68 

1 

31C 

Single  Channel  Radio  Operator8 

35 

41 

58 

1 

638 

Light  Wheel  Vehicle  Mechanic® 

31 

84 

15 

1 

88M 

Motor  Transport  Operator0 

25 

80 

•20 

0 

71L 

Administrative  Specialist 

40 

64 

35 

1 

91A/B 

Medical  Special ist/Medlcal  NCO® 

37 

59 

41 

<1 

95B 

Military  Police 

26 

34 

66 

<1 

£  Initially  tensed  Radio  Teletype  Operator. 

®  Initially  tensed  Vehicle  and  Generator  Mechanic. 

«  Initially  MOS  64C. 

d  Some  MOS  91A  soldiers  will  change  to  MOS  91B  when  they  are  promoted  to 
Skill  Level  2. 


part  of  the  performance  criterion.  Defining  a  cohort  In  this  time-oriented 
wav  appears  to  be  the  only  feasible  way  of  capturing  a  sample  with  which  to 
validate  prediction  of  NCO  performance  after  the  first  reenllstment  and 
before  the  second  tour  ends. 

Thus  the  cohort  for  the  first  longitudinal  follow-up  in  the  summer  of 
1988  is  specified  to  consist  of  soldiers  In  the  Batch  A  MOS  who  were 
previously  tested  on  predictors  and  first-tour  measures  In  the  Concurrent 
Validation,  have  3  to  5  years'  time  in  service,  and  are  still  in  the  same 
MOS  in  which  they  were  previously  tested  (excepting  the  91A  who  may  be  91B). 
For  the  second  longitudinal  follow-up,  the  cohort  will  be  comprised  of 
soldiers  in  the  Batch  A  MOS  who  entered  the  Army  In  86/87,  have  3  to  5  years 
In  service  in  the  same  MOS  in  which  they  received  initial  training,  and  are 
currently  being  tested  on  the  Experimental  Predictor  Battery  and  the  train¬ 
ing  knowledge  and  performance  measures;  they  will  be  assessed  with  the 
first-tour  criterion  measures  of  job  performance  in  the  summer  of  1988,  and 
reenlistees  are  to  be  assetsed  with  the  second-tour  performance  measures  in 
1991. 
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THE  JOB  ANALYSIS  PROCESS  FOR  THE  SECOND  TOUR 


During  the  first  three  years  of  Froject  A,  significant  time  and  effort 
were  devoted  to  gathering  job  analysis  information  for  first-tour 
performance  In  each  of  the  nine  selected  MOS  (C.  H.  Campbell,  R.  C. 
Campbell,  Ruasey,  &  Edwards,  1986;  Toquam  et  al.,  1986).  Sources  of 
information  Included  the  Soldier's  Manual  of  Common  Tasks  -  Skill  Level  1. 
the  MOS-specific  Soldier's  Manuals,  the  data  from  the  MOS-specific  Army 
Occupational  Survey  Programs  (AQSP),  generation  and  content  analysis  of 
critical  incidents  by  MOS  subject  matter  experts  (SHE),  and  Army  personnel 
management  records  and  policies.  The  job  Information  was  used  to  develop  a 
variety  of  first-tour  performance  measures:  hands-on  job  sample  tests, 
written  job  knowledge  tests,  behavioral  sunaary  rating  scales,  self-report 
questionnaires  regarding  administrative  actions,  and  questionnaires 
regarding  recency  and  frequency  of  task  performance  (C.  H.  Campbell  et  al., 
1987). 


By  Army  policy,  every  soldier  is  responsible  for  the  ability  to  perform 
all  tasks  at  lower  skill  levels,  as  well  as  the  tasks  at  the  current  skill 
level.  As  stated  In  AR  611-201,  Enlisted  Career  Management  Fields  and 
Military  Occupational  Specialties: 

"If  two  or  more  skill  levels  are  authorized  for  use  with 
an  MOS,  they  are  cumulative  in  nature."  (para  l-8a(2), 

P-  7) 


"Skill  level  identifies  skills,  proficiency,  or  ability 
ty  p  "i  Coiiy  i  cijij  1 1  cu  iui  SuCuCSSiul  pCrf  0  r  Imm  m  c  c  at  the 

grade  with  which  the  skill  level  is  associated.  There 
is  a  direct  relationship  between  grade  and  skill  level, 
without  regard  to  nonsupervisory  and  supervisory 
skills."  (para  l-8c,  p.  7) 


"The  soldier  must  be  able  to  perform  all  tasks  below  the 
skill  level  of  his  or  her  current  grade."  (para  l-9d(l), 
p.  7) 

Thus,  the  first-tour  job  analyses  could  be  used  as  a  starting  point  for  the 
second-tour  analyses,  but  expanded  performance  requirements  for  seccnd-tour 
soldiers  made  further  job  analysis  necessary.  Comparisons  of  source 
material  showed  that  job  content  for  each  job  is  different,  to  some  degree, 
when  first  tour  is  compared  to  second  tour.  For  all  but  two  of  the  Batch  A 
MOS,  tasks  were  added  to  the  job  task  domain  set  forth  in  the  MOS-specif ic 
manuals  at  Skill  Level  2;  for  every  MOS,  more  common  tasks  were  added. 

Also,  new  versions  of  the  AOSP  had  been  fielded  since  the  first-tour 
analysis  was  made  and  were  available  for  most  MOS.  Finally,  when  they 
become  NCOs,  all  soldiers  assume  various  supervisory/leadership  responsibil¬ 
ities  simply  by  virtue  of  their  rank. 
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Supervision/leadership  was  the  area  of  greatest  relevance  in  the 
second-tour  job  analysis.  The  available  literature  (Hebein.  Kaplan,  Miller, 
Olastead,  &  Sharon,  1984;  Wallis,  Korotkin,  Yarkin-Levin,  Scheoiaer,  ft 
Mumford,  1986)  indicated  that  Skill  Level  2  soldiers  have  supervisory  as 
well  as  technical  job  requirements.  The  Primary  Leadership  Development 
Course,  required  for  all  Skill  Level  2  soldiers,  includes  99  hours  of 
training  (al»o$t  half  of  the  course  hours)  in  areas  such  as  leadership, 
communications,  resource  management,  and  training  management,  which  are 
explicitly  leader-oriented  (U.S.  Army  Sergeants  Major  Academy,  1984).  The 
remaining  course  time  covers  areas  of  •Military  Studies,"  where  the  emphasis 
is  on  supervision  and  leading  troops  in  the  field.  Throughout  the 
documentation,  the  students  are  referred  to  as  'Junior  Leaders." 

To  capture  both  the  technical  and  the  supervisory  aspects  of  an  MOS, 
Project  A  staff  used  four  methods  for  second- tour  job  analysis: 

•  Task-Based  Job  Analysis  --  This  approach,  which  had  been 
used  in  the  first-tour  job  analysis,  relied  heavily  on 
existing  job  information  for  Skill  Level  2,  to  identify  a 
population  of  second-tour  technical  tasks  for  each  Batch  A 
MOS. 

•  Definition  of  Supervisory/Leadershic  Components  —  This 
approach  drew  on  task  information  developed  during  current 
Army  research  on  the  nature  of  supervision  and  leadership. 

Tasks  from  this  analysis  were  added  to  the  list  of  technical 
tasks  originating  in  the  task-based  job  analysis.  (This 
combined  list  was  then  used  as  the  pool  from  which  to 
select  a  group  of  representative  tasks  for  criterion 
measurement. ) 

•  Critical  Incident-Based  Task  Analysis  --  This  approach, 
which  also  was  used  in  the  first-tour  job  analyses,  made 
use  of  critical-incident  workshops  in  which  job  experts 
generated  examples  of  good,  poor,  and  average  second-tour 
performance,  on  both  an  Army-wide  and  an  MOS-specific 
basis.  These  were  then  clustered  into  dimensions,  for 
use  in  modifying  performance  rating  scales  developed  as 
part  of  first-tour  tests, 

•  Job  Analysis  Interviews  --  Small  groups  of  senior  NCOS 
were  interviewed  to  obtain  information  about  the  frequency 
and  importance  of  various  technical  and  supervisory 
activities  of  junior  NCOs  and  to  provide  background  to 

the  Project  A  staff  on  the  nature  of  supervisory  activities 
and  responsibilities. 
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For  each  of  the  nine  Batch  A  jobs,  definition  of  the  second-tour  job 
doiaain  began  with  the  Soldier's  Manuals  for  each  job.  Since  Skill  Level  2 
soldiers  are  held  responsible  for  all  Skill  Level  1  and  2  tasks,  the  Skill 
Level  1  task  lists  prepared  for  the  first-tour  job  analyses  were  augmented 
by  Skill  Level  2  tasks  fro*  the  higher-level  sanuals.  Soldier's  Manuals, 
prepared  by  Army  agencies  for  every  job  and  every  skill  level  within  the 
job,  list  not  only  the  tasks  required  but  also  the  conditions  under  which 
the  soldier  should  be  able  to  perfora  thea,  and  the  steps  required  for 
performance.  The  Army  also  expects  soldiers  to  be  proi icient  on  the  tasks 
in  the  Soldier's  Manuals  of  Common  Tasks,  which  likewise  include  tasks, 
conditions,  and  steps  for  basic  soldiering  tasks  (tasks  such  as  nap  reading, 
basic  first  aid,  and  operation  of  individual  weapons)  at  each  skill  level. 

Data  from  AQSPs  were  also  used  to  specify  the  technical  task  domains. 
These  surveys,  which  list  hundreds  of  task  statements  for  each  job,  are 
administered  periodically  by  the  Army  to  representative  samples  of  soldiers 
at  every  skill  level  of  each  job.  Analyses  of  the  data  include  the 
percentage  of  soldiers  at  each  skill  level  who  report  that  they  perform  the 
tasks.  The  project  staff  screened  the  lists  to  eliminate  statements  of 
tasks  not  performed  by  Skill  Level  2  soldiers,  and  then  matched  the 
remaining  AOSP  statements  with  the  task  lists  from  the  Soldier's  Manuals. 

Any  AOSP  statements  that  could  not  be  matched  with  Soldier's  Manual  tasks 
were  added  to  the  population  of  tasks  for  that  MOS.  If  Skill  Level  3  or  4 
tasks  were  performed  by  a  significant  number  of  Skill  Level  2  soldiers,  they 
were  considered  to  be  a  part  of  the  job  domain. 

After  the  second-tour  task  population  was  specified,  project  staff 
visited  the  Army  agencies  responsible  for  training  and  doctrine  in  each  MOS 
and  asked  them  to  review  the  completeness  and  accuracy  of  the  list.  Each 
proponent  was  also  asked  to  indicate  whether  any  of  the  tasks  were  likely  to 
be  eliminated  soon  because  of  equipment  or  doctrine  changes,  or  whether 
other  tasks  should  be  added  for  similar  reasons.  After  the  proponent 
completed  review  and  concurred  on  doctrinal  accuracy,  the  domain  description 
of  technical  tasks  was  considered  to  be  complete.  (The  process  parallels 
that  used  In  defining  tne  bxill  Level  1  domains,  described  in  C.  H. 

Campbell,  et  al.,  1986.)  The  total  technical  task  domains  in  the  nine  Batch 
A  MOS  ranged  between  153  and  409  tasks  each,  with  an  average  of  26C. 

For  Skill  Level  1,  job  experts  had  sorted  the  tasks  Into  clusters  based 
on  similarity  of  task  content.  Because  Army  jobs  are  hierarchical  and 
cumulative  (i.e.,  soldiers  at  higher  skill  levels  are  responsible  for 
proficiency  on  tasks  at  all  lower  skill  levels),  the  task  clusters  that  had 
been  developed  for  Skill  Level  1  were  used  as  e  starting  point  for  Skill 
Level  2.  The  new  second-tour  tasks  were  sorted  into  these  same  clusters  by 
the  project  staff.  Where  no  clusters  of  Skill  Level  1  tasks  were  similar  to 
the  new  second-tour  tasks,  new  clusters  were  formed. 

To  aid  in  subsequent  selection  of  a  smaller  sample  of  critical  tasks  to 
represent  each  domain,  judgments  of  task  criticality  and  performance 
difficulty  were  needed.  However,  concurrently  with  the  technical  domain 
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definition  just  described,  the  staff  were  also  working  on  defining  the 
components  of  supervision/le&dershlp,  as  reported  in  the  following  section. 
Once  defined,  the  technical  and  supervisory  lists  were  merged  and  judgments 
of  task  importance  and  difficulty  were  collected.  This  process  is  described 
following  the  section  on  the  supervisory/leadership  components. 


Analysis  of  Supervisory/ltadership  Components 

At  the  same  time  that  the  technical  task  descriptions  were  being 
developed  for  each  HQS,  research  on  conceptualizing  supervisory/ leadership 
activities  and  responsibilities  was  also  proceeding.  The  description  of  the 
supervision/leadership  components  took  advantage  of  both  previous  and 
current  Army  research.  An  earlier  project  had  developed  the  Supervisory 
Responsibility  Questionnaire,  based  on  critical  Incidents  describing  work 
relationships  between  first-term  soldiers  and  their  NCO  supervisors,  and  a 
current  effort  is  using  a  comprehensive  questionnaire  checklist  known  as  the 
Leader  Requirements  Survey. 

The  Supervisory  Responsibility  Questionnaire 

The  Supervisory  R  sponsibillty  Questionnaire  was  a  product  of  previous 
Project  A  research,  which  examined  the  effects  of  supervision  on  first-tour 
soldier  performance  (Hough,  Gast,  White,  &  McCloy,  1986;  White,  Gast,  & 
Rumsey,  1985).  As  part  of  this  research,  critical  incidents  had  been 
collected  to  Identify  supervisory  behaviors  that  influence  the  performance 
of  first-tour  soldiers  (i.e.,  subordinates). 

The  incidents  were  written  by  80  job  experts  from  five,  of  the  Project  A 
target  jobs.  These  experts  were  asked  to  provide  examples  of  how  Army 
supervisors  had  been  particularly  effective  or  ineffective  In  Influencing 
subordinates.  In  all,  they  generated  more  than  400  examples.  Next,  a 
retranslation  was  conducted  in  which  a  second  group  of  31  job  experts,  who 
were  familiar  with  Army  leadership  requirements,  were  asked  to  classify  the 
examples  into  a  modified  version  of  Yukl's  13-dimension  taxonomy  of 
supervisory  behavior  (Yukl,  1987).  As  a  result  of  retranslation,  9  of  the 
13  modified  Yukl  dimensions  remained. 

The  Supervisory  Responsibility  Questionnaire  was  constructed  from  a 
subset  of  these  Incidents  and  their  respective  categories.  First,  all 
incidents  that  were  not  reliably  categorized  into  a  single  dimension  were 
eliminated,  as  were  multiple  Incidents  referring  to  a  single  task  or 
behavior.  The  incident  list  was  further  reduced  by  excluding  incidents  not 
describing  a  specific  task  (e.g.,  "The  soldier  fell  asleep  while  on  guard 
duty.  [The  supervisor)  walked  up  to  the  sleeping  soldier  and  scared  him.**). 
In  the  end,  a  total  of  34  behavioral  statements  'e.g.,  Recommended  soldiers 
for  promotion)  were  written  to  represent  eight  of  Yukl's  original  categories 
(e.g.,  Recognize  and  Reward).  No  statements  were  written  for  one  category, 
Act  as  Role  Model,  because  the  incidents  grouped  under  that  category  were 
not  rich  enough  to  extract  supervisory  tasks. 


140 


Another  source  of  supervisory/leadership  information  was  the  Leader 
Requirements  Task  Analysis  Survey  (Steinberg,  1987),  which  was  developed  to 
provide  the  Army  with  information  about  the  leadership  job  requirements  of 
Army  commissioned  and  noncoonslssioned  officers  (second  lieutenant  through 
colonel,  and  sergeant  through  command  sergeant  major).  It  contains  items 
that  cover  the  leadership  domain  of  all  these  organizational  levels. 

This  task  list  was  constructed  through  an  iterative  interview  strategy. 
Several  hundred  Interviews  were  conducted  --  typically  with  six  to  eight 
military  experts  at  a  time,  and  lasting  approximately  90  minutes.  Inter¬ 
viewees  were  asked  to  describe  their  job,  focusing  on  what  they  did  to 
Influence  others  to  accomplish  their  mission  (i.e.#  the  Army  definition  of 
military  leadership  as  documented  in  FM  22-100),  and  especially  on  those 
leadership  tasks  that  differentiated  their  jobs  from  those  performed  by 
others  in  higher  or  lower  ranks  than  themselves,  or  in  different  Career 
Management  Fields.  To  ensure  that  the  resulting  task  list  both  encompassed 
the  domain  of  military  leadership  and  was  worded  in  terms  commonly  employed 
by  job  incumbents,  each  successive  group  of  experts  was  shown  the  task  list 
developed  by  the  previous  groups  and  asked  to  comment.  These  iterative 
interviews  were  conducted  until  new  groups  no  longer  added  new  tasks  and 
were  comfortable  with  the  wording  of  tasks  already  collected.  Consensus  on 
the  final  list  of  tasks  was  reached  by  a  review  committee  representing  the 
Center  for  Army  Leadership,  the  U.S.  Army  Sergeants  Major  Academy,  and  the 
Army  Research  Institute. 

The  resulting  Leader  Requirements  Survey  task  list  contains  tasks  in 
four  broad  categories  --  Train,  Teach,  and  Develop  (146  tasks);  Motivate 
(170  tasks);  Manage  (86  tasks);  and  Provide  Direction  (158  tasks)  —  for  a 
total  of  560  tasks.  In  the  present  research,  25  of  the  tasks  in  the 
category  "Provide  Direction,"  coming  under  the  subheading  of  "Provide  Input 
for  the  Direction  of  the  Larger  Organization,"  were  not  used  because  they 
were  more  applicable  to  higher-level  commissioned  officers.  (See  Steinberg, 
1987,  and  Steinberg,  van  Rijn,  &  Hunter,  1986,  for  more  information  on  the 
Leader  Requirements  Survey.) 


Combining  Data  From  the  Two  Questionnaires 
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To  determine  which  of  the  task*  in  the  Supervisory  Respons .  - 

Questionnaire  and  which  of  the  activities  on  the  Leader  Requirements  Survey 
task  list  should  be  a  part  of  the  job  content  for  second-tour  soldiers,  both 
questionnaires  were  administered  to  NCOs  (E6  and  E?)  in  the  nine  Batch  A 
M0S.  Approximately  125  NCOs  received  the  Supervisory  Responsibility 
Questionnaire  and  50  NCOs  received  the  Leader  Requirements  Survey.  For  each 
questionnaire,  the  NCOs  were  asked  to  indicate  the  importance  of  each  task 
for  performance  in  the  E5's  job.  On  the  Supervisory  Responsibility  Ques¬ 
tionnaire  they  were  also  asked  to  indicate  how  often  each  task  is  performed, 
(Gast,  Campbell,  Steinberg,  &  McGarvey,  1987.) 
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Analysis  of  the  Supervisory  Responsibility  Questionnaire  data  confirmed 
that  all  the  34  tasks  were  sufficiently  applicable,  across  a  variety  of  Army 
jobs,  to  be  retained  as  part  of  the  second-tour  domain.  The  Leader  Require¬ 
ments  Survey  data  were  used  to  select  tasks  that  more  than  half  of  the 
respondents  considered  absolutely  essential  to  the  E5's  job.  Additional 
highly  rated  tasks  were  also  selected  fro®  any  of  the  19  Leader  Requirement 
Survey  categories  and  subheadings  not  already  represented  by  at  least  two 
tasks;  ultimately,  two  subheadings  were  eliminated  because  they  failed  to 
meet  the  importance  criteria.  By  this  process,  53  tasks  were  selected  from 
the  Leader  Requirements  Survey  to  be  considered  for  the  job  domains. 

Content  analysis  of  the  two  task  lists  resulted  in  a  single  list  of  46 
tasks  that  incorporated  all  of  the  activities  on  both  lists.  Those  46  tasks 
included  the  34  task  .  from  the  Supervisory  Responsibility  Questionnaire 
(which  also  covered  many  of  the  Leader  Requirements  tasks);  eight  of  the  53 
Leader  Requirements  Survey  tasks  which  were  not  on  the  Supervisory 
Responsibility  list;  and  four  new  task  statements  prepared  to  cover  two  or 
more  Leader  Requirements  Survey  subheadings  each. 

The  46  task  statements  were  further  examined  by  reference  to  the 
categories  used  for  the  original  Supervisory  Responsibility  Questionnaire. 
Eight  categories  evolved  for  the  46  tasks. 


Merging  of  Technical  and  Supervisorv/Leadershlp 
Task  Lists  for  Measurement  Sample 

The  task  lists  derived  from  the  first  two  job  analysis  methods  -- 
technical  tasks  and  supervisory/leadership  components  --  were  to  be  merged 
and  tnen  used  as  a  population  from  which  to  select  representative  tasks  that 
would  eventually  be  used  to  measure  second-tour  job  performance.  Selection 
of  tasks  for  measurement  was  to  be  based  on  task  characteristics  —  impor¬ 
tance,  difficulty,  and  variability  --  along  with  performance  frequency  (from 
AOSP)  and  a  task  cluster  analysis  of  job  similarities. 

Judging  Task  Importance  and  Difficulty 

As  a  first  step  in  choosing  the  tasks  to  be  used  for  criterion  measure¬ 
ment.  the  Army  agency  responsible  for  each  MQS  was  asked  to  designate  15  job 
experts  --  officers  or  NCOs  in  that  military  specialty  who  b  cent  fieia 
experience  supervising  E5s  in  the  job.  These  job  experts  rateu  the  impor¬ 
tance  and  difficulty  of  tasks  for  a  hypothetical  E5  soldier  who  had  between 
3  and  5  years  of  service. 

For  the  task  importance  judgi.«nts,  the  experts  were  given  one  of  three 
scenarios,  and  asked  to  rate  (on  a  5-point  scale)  the  importance  of  the  task 
in  accomplishing  the  unit's  mission  under  that  scenario.  The  three  scenar¬ 
ios  described  either  combat  conditions  (European,  non-nuclear),  increasing 
tensions  (European,  with  a  high  state  of  training  and  strategic  readiness, 
but  short  of  actual  conflict),  or  a  garrison  environment  (state-side,  with 
training  as  the  primary  activity  and  mission).  In  all,  10  SME  ratings  for 
each  scenario  combination  were  collected,  for  a  total  of  30  sets 
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of  ratings  per  job.  The  importance  ratings  were  averaged  across  the  10 
experts  in  each  rating  condition  to  yield  three  importance  scores. 

For  the  task  difficulty  judgments,  the  experts  were  asked  to  sort  a 
"typical"  group  of  10  hypothetical  soldiers  into  five  performance  levels, 
designating  how  many  of  the  10  could  perform  the  task  all  of  the  time  (5)  to 
none  of  the  time  (1).  Task  difficulty  was  then  computed  as  the  mean  of  the 
distribution  of  the  10  soldiers,  averaged  across  the  experts.  Task  perform¬ 
ance  variability  was  computed  as  the  standard  deviation  of  the  distribution 
of  the  10  soldiers,  averaged  across  experts. 

For  the  cluster  analysis,  the  supervisory/leadership  tasks,  in  eight 
clusters,  were  added  to  the  Skill  Level  2  job  task  list  from  the  task-based 
Job  analysis.  Tasks  already  on  that  list  that  were  similar  in  content  to 
the  46  supervisory/leadership  tasks  were  subsequently  clustered  with  those 
tasks  in  the  eight  supervisory  clusters. 

Sampling  Tasks  for  Heasureoent 

To  select  a  sample  of  tasks  for  measurement ,  the  Army  agencies  for  each 
M0S  were  asked  to  provide  six  job  experts  with  recent  field  experience.  One 
Project  A  staff  member  also  served  on  the  task  selection  panel.  The  infor¬ 
mation  to  be  considered  included  the  task  content  itself;  the  importance 
rating  for  each  task;  the  performance  difficulty  and  expected  performance 
variability  for  each  task;  the  frequency  of  task  performance  as  shown  by  the 
A0SP  analyses;  and  the  task  cluster  membership  for  each  task. 

The  panel  was  to  select  45  tasks  for  each  job,  30  technical  tasks  and 
15  supervisory  tasks.  The  separate  targets  were  provided  to  give  both  the 
technical  and  the  supervisory  portions  of  the  domain  independent  considera¬ 
tion.  Proportional  targets  were  also  set  for  each  cluster. 

Tasks  were  selected  by  a  modified  Oelphi  process.  In  the  first 
session,  the  experts  on  the  selection  panel  independently  selected  tasks, 
using  the  given  target  numbers  for  each  cluster.  The  picks  were  tallied  and 
presented  to  the  panel  in  the  second  session.  They  again  made  independent 
selections,  this  time  giving  reasons  for  each  of  their  picks.  The  picks 
were  again  tallied,  and  the  reasons  summarized,  for  consideration  in  the 
third  round  of  independent  selections. 

Finally,  in  the  fourth  session,  the  panel  members  met  as  a  group  to 
discuss  their  selections  and  resolve  their  differences,  until  they  arrived 
at  a  consensus  on  the  45  tasks.  During  this  final  session,  they  were 
permitted  to  violate  the  cluster  target  numbers,  but  were  still  within  the 
constraints  of  30  technical  and  15  supervisory  tasks.  The  panel  members 
then  also  assigned  rankings  to  the  tasks  to  indicate  priority  for  inclusion 
in  the  final  set  of  tasks  representing  the  jobs. 
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*Htiea1  Incident-Based  Job  Analysts 


The  content  of  performance  for  second-tour  MCS  was  initially  hypothe¬ 
sized  to  include  both  technical  and  supervisory  components  which  would  be 
incorporated  into  the  second-tour  performance  measures.  However,  it  seemed 
likely  that  the  extent  to  which  supervision  is  an  Important  part  ot  the 
second-tour  job  would  vary  across  the  different  MOS,  which  suggested  the 
possibility  that  some  supervisory  components  might  be  applicable  to  all  HOS 
while  others  might  be  MOS-specif ic. 


To  incorporate  the  Army-wide  versus  MOS-specif ic  distinction,  the 
procedures  used  to  develop  Behavioral  Summary  Scales  (Borman,  1979)  were 
applied  to  analyzing  second-tour  job  performance.  (Details  concerning  the 
sample  and  procedures  may  be  found  in  Pulakos,  et  al.,  1987.) 


The  procedures  are  based  on  an  inductive  critical  incident  analysis 
strategy  (J.  f.  Campbell,  Dunnette,  Arvey,  &  Hellervik,  1973),  which 
requires  persons  familiar  with  the  performance  demands  of  a  job  to  generate 
examples  of  effective,  mid-range,  and  ineffective  behavior  observed  on  that 
job.  Content  analysis  of  the  examples  then  yields  preliminary  dimensions  of 
performance,  and  an  independent  retranslation  of  the  examples  into  the 
dimensions  provides  a  way  of  checking  on  the  clarity  of  the  individual 
critical  incidents  and  the  content  validity  of  the  dimension  system.  This 
is  accomplished  by  asking  persons  familiar  with  the  target  domain  to  make 
two  judgments  about  each  incident  example:  the  dimension  or  category  to 
which  it  belongs,  mased  on  its  content,  and  the  effectiveness  level  It 
reflects.  If  disagreement  occurs  on  either  category  membership  or  rated 
Effectiveness  level  the  incident  ssay  be  unclear,  in  form  and/or  substance, 
an(/should"be  revised  or  eliminated/  Also,  confusion  between  two  or  more 
categories  in  the  sorting  of  several  examples  may  reflect  a  poorly  formed 
category  system. 


frrmv-Wide  Critical  Incident  Analysis. 

In  a  series  of  workshops,  the  participants  were  asked  to  generate 
examples  of  what  they  considered  to  be  the  performance  domain  of  second-tour 
soldiers.  A  tctal  of  1,000  critical  incidents  was  generated  by  172  officers 
and  NCOS.  These  incidents  were  edited  to  a  common  format  and  then  content 
analyzed  to  form  12  preliminary  dimensions  of  second-tour  Army-wide  perfor¬ 
mance. 


The  Army-wide  performance  categories  that  had  been  developed  for  the 
first-tour  soldiers  (Pulakos  &  Borman,  1986)  were  also  found  in  the  analysis 
for  second-tour  soldiers.  In  addition,  three  generic  supervisory  dimensions 
emerged  from  the  content  analysis  of  the  incidents,  which  suggested  that 
second-tour  soldiers  do,  in  fact,  perform  most  of  the  work  that  first-tour 
soldiers  perform  and  also  supervise  that  work. 

The  retranslation  judges  were  a  group  of  81  officers  and  HCOs,  none  of 
whom  had  participated  in  the  workshops  to  generate  the  critical  incidents. 

To  accomplish  the  retranslation,  judges  were  provided  with  definitions  of 
the  12  dimensions  and  a  7-point  effectiveness  scale  (where  1  •  extremely 
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Ineffective,  4  -  average,  and  7  ■  extremely  effective)  to  ^uide  their 
ratings  of  effectiveness.  To  keep  the  retranslation  exercise  within 
reasonable  bounds,  each  judge  was  asked  to  evaluate  only  250  behavioral 
incidents. 

An  initial  screening  of  the  judging  results  identified  potential  random 
responders  or  individuals  who  obviously  did  not  understand  the  retranslation 
task.  Specifically,  respondents  were  scored  on  12  critical  incidents,  each 
of  which  the  research  staff  believed  could  be  reliably  classified  into  one 
of  the  12  dimensions  by  anyone  who  understood  and  attended  to  the  task. 
Respondents  who  did  not  correctly  categorize  at  least  50  percent  of  these 
incidents  were  deleted  from  the  sample.  Using  this  criterion,  seven 
respondents  out  of  the  total  81  were  omitted  from  the  judging  sample, 
leaving  a  sample  size  of  74  respondents  for  the  retranslation  analyses. 

Two  criteria  were  adopted  for  retaining  critical  Incidents  for  develop¬ 
ment  of  scales:  (a)  more  than  50  percent  of  the  judges  sorted  the  example 
into  a  single  dimension,  and  (b)  the  standard  deviation  of  the  given 
effectiveness  ratings  for  the  example  is  less  than  1.50.  Application  of 
these  criteria  left  734  of  the  1,000  exa«$)les  (73.4%)  for  subsequent  scale 
development  efforts  (between  21  and  168  per  dimension).  The  retranslation 
results  also  indicated  that  all  12  of  the  dimensions  resulting  from  the 
initial  categorization  of  the  incidents  should  be  retained. 


HOS-Soecific  Performance  Analysis 

While  the  second-tour  Army-wide  dimensions  were  developed  using  the 
entire  sequence  of  behavioral  summary  scale  procedures,  development  of  the 
second-tour  MOS-specific  dimensions  followed  a  different  procedure.  It 
involved  a  process  for  revising  the  existing  first-tour  MOS-specific  rating 
scales  so  that  they  would  be  appropriate  for  describing  and  evaluating 
second-tour  performance. 


To  accomplish  the  revision,  a  critical  incident  analysis  workshop  was 
conducted  with  approximately  25  officers  and  NCOS  in  each  of  the  nine  taryet 
jobs  (Batch  A  MOS)  to  generate  examples  of  effective,  average,  and  ineffec¬ 
tive  second-tour  MOS-specific  job  performance.  The  procedures  previously 
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specific  workshops.  However,  rather  than  writing  examples  that  would  apply 
to  any  MOS,  participants  were  Instructed  to  write  critical  Incidents 
specific  to  their  particular  job.  The  number  of  incidents  generated  for 
each  MOS  ranged  from  58  to  236,  with  an  average  of  180. 


Comparison  of  First-  and  Second-Tour  MOS-Soecific  Incidents.  After  the 
incidents  were  edited  to  a  common  format,  they  were  categorized  by  the 
project  staff,  using  the  first-tour  MOS-specific  category  system  as  a 
starting  framework.  If  a  second-tour  incident  did  not  fit  into  an  existing 
first-tour  category,  a  new  category  was  Introduced.  This  process  made  it 
possible  to  judge  whether  the  same  or  different  categories  should  be  used 
for  evaluating  second-tour  performance.  It  also  yielded  information 
regarding  what  specific  category  additions  or  deletions  were  needed  to 
describe  the  second-tour  performance  domain  comprehensively. 
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Almost  #11  of  the  first-tour  MOS-specific  perfonBance  categories  were 
judged  to  be  appropriate  for  second-tour  MOS.  For  each  category,  the  next 
step  was  to  examine  the  content  of  the  incidents  to  determine  whether  the 
perforaance  requirements  were  appreciably  different  for  second-tour  and 
first-tour  soldiers.  This  was  an  Important  step  because,  although  the 
general  Meaning  of  the  perforaance  dlaensions  for  first  and  second  tour 
Might  be  the  saoe,  it  was  at  least  possible  that  the  diaension  definitions 
or  anchors  light  need  to  be  Modified  to  Make  the  scales  aore  precisely 
relevant  for  evaluating  second-tour  perforaance. 

For  sone  dimensions,  comparisons  indicated  that  aore  was  expected  of 
second-tou**  soldiers  than  was  expected  of  their  first-tour  counterparts. 
Under  such  circumstances,  the  summary  statement  anchors  were  Modified  to 
reflect  the  appropriate  performance  standards.  For  other  dimensions  the 
incidents  suggested  that  second-tour  soldiers  were  responsible  for  knowing 
how  to  operate  and  Maintain  more/different  pieces  of  equipment  than  were 
first-tour  soldiers.  This  distinction  also  was  incorporated  into  the 
second- tour  suamary  stateaents. 

For  several  MOS,  the  second-tour  incidents  suggested  that  MOS-specific 
supervisory  performance  categories  should  be  developed.  Accordingly, 
preliminary  summary  statement  anchors  were  written  for  these  supervisory 
dimensions.  However,  care  was  taken  not  to  duplicate  the  Army-wide  leader¬ 
ship/supervision  diaensions  as  the  MOS-specific  categories  were  intended  to 
reflect  aspects  of  supervision  that  were  relevant  only  to  the  particular 
job.  For  five  of  the  nine  MOS,  a  total  of  six  MOS-specific  supervisory 
dimensions  were  developed. 

Content  Validation.  For  each  of  the  nine  MOS,  two  scale  revision 
workshops  were  conducted  with  10-14  participants  (officers  and  NCOs)  in 
each.  These  individuals  were  different  from  those  who  generated  the 
behavior  examples.  The  purpose  was  to  have  subject  matter  experts  review 
the  proposed  second-tour  performance  categories  and  revise  the  dimension 
definitions  and  anchors  as  necessary  to  make  the  scales  appropriate  for 
evaluating  second-tour  MOS-specific  performance.  Participants  were  asked  to 
address  three  focal  questions: 

•  Do  the  diaension  anchors  contain  material  that  is  not  relevant 
for  evaluating  second-tour  soldier  effectiveness? 

e  Do  the  diaension  anchors  for  various  levels  of  effectiveness 
accurately  reflect  what  would  be  expected  of  a  second-tour 
soldier  performing  at  the  ineffective,  average,  and  effective 
levels  of  performance? 

e  Do  the  proposed  dimensions  tap  all  of  the  MOS-specific 
performance  components  of  the  second-tour  soldier's  job? 

Participants  were  asked  to  think  about  expectations  for  second-tour  perfor¬ 
mance  and  recommend  any  changes  that  would  make  the  scales  maximally 
relevant  for  evaluating  second-tour  soldiers. 
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Based  on  the  Input  froa  the  workshop  participants,  the  scales  were 
revised.  In  aost  cases,  only  ainor  wording  changes  were  Bade  to  the  summary 
statements.  In  a  few  cases,  however,  the  dimensions  themselves  as  well  as 
their  anchors  were  changed  substantially  —  usually  because  the  job  require¬ 
ments  had  actually  changed  since  the  tiae  the  first- tour  scales  were 
developed  and  the  second-tour  critical  incidents  were  collected. 

For  each  MOS  a  retranslation  workshop  also  was  conducted  with  approxi¬ 
mately  20  officers  and  NCOs.  Workshop  participants  were  again  different 
froa  those  who  generated  the  critical  incidents  and  those  who  reviewed  and 
revised  the  proposed  second-tour  rating  scales.  The  purpose  was  to  check  on 
the  intended  effectiveness  levels  of  the  behavioral  summary  statements 
anchoring  each  HOS-specific  performance  dimension,  as  well  as  to  check  on 
the  dimension  structures  theaselves. 

Rather  than  retranslating  individual  critical  incidents  (as  was  done 
with  the  Army-wide  retranslation  workshops  described  earlier),  participants 
were  asked  to  retranslate  the  actual  summary  statements  that  would  be  used 
to  anchor  the  rating  scale  dimensions.  As  noted  earlier,  three  summary 
statements  were  used  to  anchor  each  dimension:  one  describing  low-level  or 
ineffective  performance,  one  describing  middle-level  r.r  average  performance, 
and  one  describing  high-level  or  effective  performance.  Participants  were 
provided  with  definitions  of  each  dimension  and  a  booklet  containing  the 
summary  statements  listed  in  a  random  order.  They  were  asked  to  make  two 
judgments  about  each  sunsnary  statement:  the  dimension  or  category  to  which 
it  belonged  based  on  its  content,  and  the  effectiveness  level  it  represented 
from  1  (very  ineffective)  to  7  (very  effective).  The  number  of  dimensions 
for  the  different  MQS  ranged  from  7  to  14. 

Again,  an  Initial  screening  was  undertaken  to  identify  potential  random 
responders  or  individual',  who  obviously  did  not  understand  the  retranslation 
task.  For  each  MOS,  respondents  were  scored  on  approximately  10  critical 
incidents,  each  of  which  the  research  staff  believed  could  be  reliably 
classified  into  one  of  the  performance  dimensions.  If  respondents  did  not 
correctly  recategorize  at  least  50  percent  of  these  incidents,  they  were 
deleted  from  the  sample.  Of  the  193  total  participants  in  the  retranslation 
workshops,  this  procedure  led  to  the  exclusion  of  22  from  the  retranslation 
analyses  reported  below. 

For  almost  all  (98%)  of  the  summary  statements  for  all  of  the  nine  MOS, 
at  least  half  of  the  retranslation  sample  placed  them  in  the  intended 
cateoory,  and  for  92  percent  of  the  statements  more  than  75  percent  of  the 
sample  categorized  them  as  intended.  The  mean  effectiveness  level  was  also 
very  close  to  the  Intended  effectiveness  level  for  most  of  the  summary 
statements.  That  is,  if  the  statement  was  intended  to  be  a  low-level  or 
ineffective  anchor,  Its  mean  effectiveness  level  was  about  1.0.  For  those 
intended  to  be  a  middle-level  or  average  anchor,  the  mean  effectiveness 
level  was  about  4,  and  for  those  intended  to  be  a  high-level  or  effective 
anchor,  the  mean  effectiveness  level  was  about  7. 
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However,  in  a  few  statements  (about  14\  across  all  HOS)  there  was  some 
discrepancy  between  the  mean  effectiveness  level  and  the  intended  effec¬ 
tiveness  level  (i.e.,  the  effectiveness  rating  was  more  than  1.0  point  away 
from  the  Intended  effectiveness  level).  Such  statements  were  revised  to 
ensure  that  they  reflected  the  proper  effectiveness  levels. 

Comments  on  the  HOS-Speclf ic  Scale  Development.  One  lesson  learned 
from  the  MOS-specific  scale  development  effort  is  that  a  procedure  less  tine 
consuming  than  the  usual  sequence  for  developing  behavioral ly  anchored 
rating  scales  may  be  very  effective  when  rating  scales  based  on  critical 
incidents  for  a  similar  job  are  already  available.  Because  the  first-tour 
rating  scales  were  available  for  each  of  the  nine  HOS  and  because  the 
second-tour  performance  requirements  were  reported  to  be  similar  in  cany 
ways  to  first-tour  requireoents,  it  seemed  appropriate  to  simplify  the 
procedure  for  developing  the  MOS-specific  scales.  Accordingly,  tne  first- 
tour  scales  were  used  as  a  starting  point  and  those  parts  of  tne  scales 
needing  changes  were  revised  utilizing  a  relatively  small  number  of  perform¬ 
ance  examples  and  a  group  of  job  experts  working  directly  on  the  scales' 
summary  statements.  This  shortened  procedure  considerably  reduced  the  time 
and  expense  needed  to  develop  rating  scales  without  reducing  the  quality  of 
the  scales,  as  was  apparent  from  the  satisfactory  ratranslation  results  for 
the  final  summary  statements. 


job  Analysis  Interviews 


job  analysis  method  used  for  the  second-tour  investigation 
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groups  (5-8  people)  of  NCOs  in  each  of  the  nine  MOS.  They  were  asked  about 
the  number  or  percentage  of  E5s  who  would  probably  be  in  different  duty 
positions,  and  about  the  normal  activities  of  those  individuals.  They  were 
also  asked  to  indicate  how  many  hours  per  week  those  individuals  would  spend 
on  each  of  nine  supervisory  activities  and  each  of  two  general  areas  of 
actual  task  performance,  and  how  important  each  of  those  11  aspects  of  the 
job  is  for  the  second-tour  NCO.  A  copy  of  the  interview  protocol  for  one 
HOS  is  shown  in  Appendix  6. 


The  interviews  proved  to  be  of  particular  value  in  educating  the 
project  staff  about  the  nature  of  the  second-tour  job  in  each  of  the  nine 
HOS.  Information  about  the  relative  importance  and  time  spent  on  leader¬ 
ship/supervision  versus  technical  activities  supplemented  the  information 
obtained  from  the  other  job  analysis  methods. 


RESULTS  FROM  JOB  ANALYSES 

The  information  gathered  by  the  methods  described  was  summarized  in  a 
job  analysis  summary  book  for  each  MOS.  These  books  provided,  as  consoli¬ 
dated  sources,  Information  concerning  activities,  tasks,  and  critical 
behaviors  required  of  second-tour  soldiers  in  each  of  the  nine  MOS,  for  use 
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by  job  analysts  In  ensuring  coverage  of  all  aspects  of  the  jobs  in  the 
critical  domains.  Each  MQS  volume  contains  the  following  sections:  Job 
Analysis  Interview  Summary;  Second-Tour  Task  Description;  Second-Tour 
Critical  Incident  Analysis;  and  Comparison  of  Task-Based  Analysis,  Super¬ 
visory  Analysis,  and  Behavior-Based  Analysis  Results. 
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For  every  MOS,  as  defined  by  the  task-based  descriptions,  the  content 
of  the  second-tour  job  tasks  tends  to  be  more  difficult  and  more  complex 
than  for  first-tour  soldiers.  As  mentioned  earlier,  this  is  in  part  a 
result  of  the  Army's  policy  regarding  skill  level  progression:  The  soldier 
must  be  able  to  perform  all  tasks  below  the  skill  level  of  his  or  her 
current  grade  plus  the  additional  tasks  introduced  at  that  grade. 

For  the  most  part,  the  additional  second-tour  tasks  are  more  difficult 
bu%  are  of  the  same  general  content  as  the  first-tour  tasks,  as  shown  by 
comparison  of  the  first-tour  and  second-tour  cluster  structures.  Within  the 
six  noncombat  jobs,  common  task  clusters  were  significantly  realigned;  by 
contrast,  there  was  little  change  in  the  combat  jobs  (MOS  I1B  Infantryman, 
13B  Cannon  Crewman,  and  19E  Armor  Crewman).  The  addition  of  tasks  also 
caused  several  of  the  technical  clusters  to  split  into  better  differentiated 
groups  of  tasks.  The  changes  to  the  cluster  structures  are  detailed  for 
each  of  the  nine  MOS  in  Appendix  H;  an  example  for  one  MOS  is  shown  in 
Figure  5.1. 

An  important  difference  between  the  first-  and  second-tour  task  lists 
lies  in  the  expansion  of  MQS-speciflc  leadership  clusters  for  every  MOS.  In 
seven  of  the  MOS  (13B  Cannon  Crewman,  19E  Armor  Crewman,  31C  Single  Channel 
Radio  Operator,  638  Light  Wheel  Vehicle  Mechanic,  88M  Motor  Transport 
Operator,  91A/B  Medical  Specialist/Medical  NCO,  and  95B  Military  Police),  a 
new  cluster  was  formed  to  represent  tasks  involving  either  tactical  opera¬ 
tions  leadership  or  administrative  supervision.  In  the  other  two  MOS  (1 IB 
Infantryman  and  71L  Administrative  Specialist),  such  clusters  were  greatly 
expanded  due  to  the  addition  of  new  tasks. 

Examination  of  the  complete  second-tour  task  list  permits  us  to 
estimate  the  relative  proportion  of  content  in  each  Batch  A  job  that 
Involves  supervisory  responsibilities,  in  contrast  to  the  performance  of 
technical  tasks.  Although  the  number  of  tasks  judged  to  explicitly  require 
the  exercise  of  supervision  varies  widely  across  MOS.  ranging  from  48  to  129 
tasks,  the  percentage  of  tasks  that  were  judged  supervisory  in  nature  was 
very  similar  for  all  nine  MOS,  ranging  from  30  to  33  percent  of  the  total. 
Keep  in  mind  that  this  is  not  an  estimate  of  relative  criticality,  or  amount 
of  time  spent;  those  data  come  from  other  sources. 

Except  for  the  46  tasks  derived  from  the  Supervisory  Responsibility 
Questionnaire,  nearly  all  of  the  tasks  requiring  second-tour  soldiers  to 
perform  as  supervisors  are  derived  from  MOS-specific  sources.  Three  Skill 
Level  2  common  tasks  have  supervisory  responsibilities,  and  they  are  in  the 


211  Infantryman  Task  Dot  In 

Of  the  12  Skill  Level  1  task  clusters,  11  were  retained  for  second 
tour;  the  cluster  on  Maintaining  and  Operating  Vehicles  was  dropped  at  the 
request  of  the  Proponent.  All  new  second  tour  tasks  were  categorized  into 
the  11  first  tour  clusters,  therefore  no  additional  clusters  were  fonaed 
for  second  tour.  Although  clusters  nearly  always  changed  in  size  and  in  the 
specific  tasks  included,  the  general  content  of  ten  of  the  clusters  was 
unchanged. 

The  one  cluster  that  did  change  dramatically  was  the  cluster  entitled 
"Conduct  Tactical  Operations."  For  second  tour,  the  cluster  has  nearly 
twice  as  many  tasks  (22  compared  to  12),  including  11  tasks  on  supervising 
and  directing  activities  of  the  fire  team,  squad,  or  platoon,  five  tasks  or. 
leading  patrols  or  missions,  and  two  tasks  on  supervising  weapons  and  coabat 
operations.  These  are  directly  comparable  to  the  two  leadership/supervisory 
diversions  that  emerged  from  the  behavioral  performance  work. 

The  46  Supervisory  Responsibilities  Questionnaire  activities  were 
categorized  into  eight  supervision  clusters.  In  addition,  the  domain 
included  seven  MOS-specific  supervision  tasks  that  were  also  categorized 
into  those  eight  clusters.  Specifically,  four  tasks  pertaining  to  field 
reports  and  orders  were  placed  in  the  cluster  “Provide  Inforcation,"  and 
three  MGS-specific  training  tasks  are  in  the  cluster  “Train  and  Develop." 

Thus  these  two  clusters,  for  the  116  E5,  represent  sore  than  just  generic 
activities  of  passing  on  information  and  providing  training. 

\ 

The  task  clusters  for  the  UB  second  tour  domain  are  listed  on  t.h?. 
following  page,  along  with  brief  descriptions  of  the  cluster  content. 


Figure  5.1.  Example  of  Second-Tour  Task  Clusters:  MOS  llt>  Infantryman 
(Page  1  of  2  pages). 
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Task  Clusters:  115  Infantryman 


FIRST  AIO:  Diagnosing  Injuries.  administering  first  aid,  and  transporting  casualties. 


I  AND  RAVI  CAT  I OR  AND  MR  READING:  Moving  ovar  unknown  terrain,  reading  maps.  reeding  compass, 
determining  location,  direction,  and  distance. 


NUCLEAR,  SIOLOCICAL,  CHEMICAL  (NBC):  Tasks  performed  under  NBC  conditions.  Including  putting 
on  protective  mask  and  clothing,  operating  end  Maintaining  NBC  equipment,  and  reporting  BBC 
conditions. 


WEAPONS:  Operation,  maintenance,  and  positioning  of  weapons  (M16  rifle,  H60  machlnegun.  LAW, 
SAu,  grenades,  grenade  launcher.  ,45  cal  pistol,  .50  cal  machlnegun,  dragon. 


MOVE HE hr  AMD  SURVIVAL  IN  FIELD:  Tasks  related  to  battlefield  survival  In  defensive  end 
offensive  situations. 


COMMUNICATIONS:  Installation  and  operation  of  radio  and  field  telephone  equipment,  end 
common ictt ions  security  procedures. 


DETECT  AND  10ENTIFY  THREATS:  Surve1ltar.ee  tasks.  Including  search  and  scan  procedures,  and 
identifying  threat  vehicles  end  aircraft. 


HI CHTSI GhTS:  Operation  (mounting,  zeroing,  engaging  targets)  and  maintenance  of  hand-held  and 
weapor.-mounted  night  sighting  devices. 


MIMES  AMD  DEMOLITIONS:  Installing  and  disarming  sines  and  booby  traps,  and  constructing 
nonelectric  and  electric  demolition  systems. 


MOVEMENT  IN  URBAN  TERRAIN:  Tactical  citations  in  built-up  areas. 


CONDUCT  TACTICAL  OPERATIONS:  Supervising  and  directing  ac'lvltiev  of  the  fire  tean.  squad,  or 
platoon,  leading  patrols  or  missions,  supervising  weapons  and  combat  operations. 


PLAN.  Ob  LAN!  2E ,  AMO  KONl'OR:  Assigning  work  tasks,  supervising  performance  of  tasks, 
conducting  Inspections,  and  monitoring  equipment  condition  and  supplies. 


CLARIFY  ROLEF.  PROVIDE  FEEDBACK:  Performance  monitoring  and  counseling  of  suburol nates. 


PROVIDE  iNFORKAT ION : 


rtusiiiq  fit 
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RECOGNIZE.  REWARD:  Providing  formal  and  informal  rewards  and  recognition  for  good 
performance,  reeoawend !ng  soldiers  for  promotion  or  awards. 


TRAIN.  DEVELOP;  Planning  and  conduct  Ing  Individual  and  teat*  training,  providing  career 
counseling,  and  providing  opportunities  for  leadership. 


SUPPORT:  Listening  to  subordinates*  personal  problems,  and  counseling,  assisting,  or 
arranging  assistance,  as  appropriate. 


DISCIPLINE,  PUNISH:  Providing  formal  or  Informal  disciplinary  measures  to  subordinates. 
ACT  AS  MODEL:  Setting  the  example  for  subordinates. 

Figure  5.1.  Excwple  of  Second-Tour  Task  Clusters:  NOS  11B  Infantryman 
(Page  2  of  2  pages). 
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domain  for  every  MOS.  Additionally,  many  tasks  that  are  derived  fro*  MQS- 
specific  sources  (Soldier's  Manuals  and  AOSP)  appear  in  core  than  one  of  the 
MOS. 
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Critical  Incident-Based  Job  Analysis 

As  mentioned  previously,  analysis  of  the  Army-wide  critical  incidents 
generated  by  the  workshops  led  to  the  addition  of  three  dimensions  reflect 
ing  increased  supervisory/leadership  responsibilities  across  all  Jobs. 
These  three  dimensions  in  effect  replaced  a  single  first-tour  leadership 
dimension.  All  nine  of  the  other  Army-wide  dimensions  that  had  been 
developed  for  first-tour  soldiers  were  replicated  for  the  second-tour  job. 
Thus  the  Army-wide  performance  behaviors  which  were  considered  to  be 
important  for  first-tour  soldiers  were  also  judged  to  be  components  of 
effective  performance  for  second-tour  soldiers,  regardless  of  MOS.  The  12 
Army-wide  performance  dimensions  are  shown  in  Table  5.2. 


Table  5.2 

Second-Tour  Army-Wide  Dimensions 


A.  Displaying  Technical  Knowledge/Skill 

B.  Displaying  Effort,  Conscientiousness,  and  Responsibility 

*C.  Organizing,  Supervising,  Monitoring,  and  Correcting  Subordinates 
*D.  Training  and  Developing 

*E.  Showing  Consideration  and  Concern  for  Subordinates 

F.  following  Regulations/Orders  and  Displaying  Proper  Respect  for 
Authority 

G.  Maintaining  Own  Equipment 

H.  Displaying  Honesty  and  Integrity 

! .  Maintaining  Proper  Physical  Fitness 

0.  Developing  Own  Job/Soldiering  Skills 

K.  Maintaining  Proper  Military  Appearance 

l.  Controlling  Own  Behavior  Related  to  Personal  Finances, 
Drugs/Alcohol,  and  Aggressive  Acts 


*New  >eadcrsh1p/supervisory  dimensions  for  second  tour. 
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Analysis  of  the  MOS-specific  critical  incidents  and  subsequent  retrans¬ 
lation  suggested  that  all  but  two  of  the  first-tour  dimensions  be  retained 
(one  dimension  was  dropped  from  the  MOS  71L  Administrative  Specialist  as 
being  too  low  level  for  second-tour  soldiers,  and  cne  dimension  was  dropped 
for  MOS  95B  Military  Police  becausa  it  was  covered  by  other  dimensions).  In 
three  cases,  a  single  first-tour  dimension  was  split  into  two  dimensions  for 
second  tour. 

Of  the  85  first-tour  MOS-specific  dimensions,  38  (45V)  were  unchanged 
for  second  tour,  except  for  minor  wording  changes  for  clarification.  The 
added  technical  and  supervisory  responsibilities  for  secor.d-tour  soldiers 
resulted  in  substantial  changes  to  44  (52V)  of  the  dimensions.  As  mentioned 
previously,  MOS-specific  supervisory  dimensions  were  developed  for  five  of 
the  nine  MOS;  the  names  of  the  second-tour  supervisory  performance  dimen¬ 
sions  by  MOS  are  shown  in  Table  5.3.  Detailed  summaries  of  the  changes  to 
the  dimensions  structure  for  second-tour  are  given  for  each  MOS  in  Appendix 
I;  an  example  for  one  MOS  is  shown  in  Figure  5.2. 


Table  5.3 

Supervisory  Performance  Categories  for  Second-Tour  NOS-Specific  Scales 


MOS 

Performance  Category  Name 

1  IB 

Infantryman 

Supervising  Soldiers  in  the  Field 

Leading  the  Team 

13B 

Cannon  Crewman 

None 

19E 

Armor  Crewman 

Assuming  Supervisory 

Responsibilities  in 

Absence  of  Tank  Commander 

31C 

Single  Channel  Radio  Operator 

Managing  the  RATT  Rig 

63B 

light  Wheel  Vehicle  Mechanic 

Checking  Repairs  Made  by 

Other  Mecnanics 

88M 

Motor  Transport  Operator 

None 

71L 

Administrative  Specialist 

Hone 

91A/B  Medical  Specialist/Medical  NCO 

None 

95B 

Military  Police 

leading  the  Team  in  a 

Tactical  Environment 
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The  content  of  six  of  the  12  first  tour  performance  categories  wa s 
unchanged  for  second  tour.  Five  categories  were  modified  to  reflect 
additional  performance  requirewents/expectations  (such  as  maintaining 
equipment  even  when  not  specifically  told  to  do  so,  being  able  to 
demonstrate  use  of  weapons  in  addition  to  being  able  to  use  the  weapons 
expertly,  and  supervising  subordinates  in  guard  positions.) 

The  greatest  difference  between  first  and  second  tour  responsibilities 
is  evidenced  by  the  critical  Incidents  pertaining  to  leadership 
responsibilities.  The  first  tour  category  "assisting  and  Leading  Others" 
was  divided  into  two  second  tour  categories:  "Leading  the  Team"  and 
"Supervising  Soldiers  ir.  the  Field."  These  two  categories  include 
responsibilities  for  ensuring  that  troops  have  the  required  supplies  and 
equipment,  ensuring  the  safety  and  well-being  of  soldiers,  briefing  troops 
about  the  mission,  ensuring  that  work  is  properlv  completed,  using  sound 
judgment  to  accomplish  the  mission,  and  leading  by  example. 

The  first  and  second  tour  performance  category  names  are  shown  below; 
the  second  tour  performance  category  definitions  are  listed  on  the  next 
page. 


FIRST  TOUR  PERFORMANCE  CATEGORIES 

Maintaining  Supplies,  Equipment, 
and  Weapons 

Assisting  and  Leading  Others 

Navigation 

Use  of  Weapons  and  Other  Equipment 

Field  Sanitation,  Personal  Hygiene 
and  Personal  Safety 

Fighting  Positions 

Avoidinj  Enemy  Detection 

Operating  a  Field  Phone/Radio 

Reconnaissance  and  Patrol 

Guard  and  Security  Duties 

Prisoners  of  War 

Courage  and  Proficiency  in  Battle 


SECOND  TOUR  PERFORMANCE  CATEGORIES 

Maintaining  and  Accounting  for 
Weapons  and  Equipment 

Supervising  Soldiers  In  the  Field 

Leading  the  Team 

Navigation 

Use  of  Organic  Weapons  and  Equipment 

Field  Sanitation,  Personal  Hygiene, 
and  Personal  Safety 

Fighting  Positions 

Avoiding  Enemy  Detection 

Operating  a  Radio  Set 

Reconnaissance 

Guard  and  Security  Duties 

Prisoners  of  War 

Proficiency  in  Battle 


Figure  5.2.  Example  of  Performance  Dimension  Changes  for  Second  Tour: 
KOS  11B  Infantryman  (Page  1  of  2  pages). 
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SECOND  TOUR  NOS  PERFORMANCE  CATE60RY  NAMES  AND  DEFINITIONS  FOR  NOS: 

INFANTRYMAN  (11B) 

j 

A.  Maintaining  and  Accounting  for  Equipment  end  Weapons 

Nor  effective  It  tech  soldier  In  ensuring  that  ell  equipment  end  weapons  ere  well 
maintained  end  available  for  use  In  the  field? 

B.  Supervising  Soldiers  in  the  Field 

How  effective  is  each  soldier  in  ensurino  the  troops  have  necessary  supplies/ 
equipment  and  ensuring  the  safety  and  well-being  of  soldier? 

C.  Leading  the  lean 

Now  effective  is  each  soldier  when  leading  a  teas  in  a  field  environment? 

0.  Navigation 

Now  effective  is  etch  soldier  in  using  navigational  equipment  and  navigating  In 
the  field? 

I.  Use  of  Organic  Weapons  and  Equipment 

How  effective  it  each  soldier  in  using  organic  weapons  and  equipment  safely  and 
prof tciently? 

f.  Field  Sanitation,  Personal  Hygiene,  and  Personal  Safety 

How  effective  1$  etch  soldier  in  maintaining  sanitary  conditions,  personal 
hygiene,  and  personal  safety  in  the  field? 

C.  f ighttng  Positions 

How  effective  is  each  soldier  in  preparing  a  fighting  position,  range  cards,  and 
sector  sketches? 

H.  Avoiaing  Enemy  Oeieciion 

Hew  effective  it  each  soldier  in  avoiding  enemy  detection  during  movement  and  in 
established  defensive  positions  while  in  the  field? 

I .  Operating  a  Radio  Set 

How  effective  it  each  soldier  in  putting  a  radio  into  operation  and  using  it 
proper ly? 

J.  Reconn* itsanc* 

How  effective  is  each  soldier  in  performing  reconnaissance  activities? 

K.  Cuard  and  Security  Outlet 

How  effective  It  each  soldier  in  performing  sergeant  of  the  guard  and  security 
duties  and  manning  observation  pests? 

L.  Prisoners  of  War 

How  effective  is  each  soldier  In  guarding  and  processing  prisoners  of  war  during 
field  exercises  or  In  combat? 

H.  Proficiency  in  Battle 

How  effective  1$  each  soldier  In  demonstrating  proficiency  in  engaging  the  enemy 
during  field  exercises  or  In  combat? 


Figure  S.2.  Example  of  Performance  Dimension  Changes  for  Second  Tour 
NOS  13 B  Infantryman  (Page  2  of  2  pages). 


Thus,  although  the  Batch  A  MQS  vary  in  the  extent  to  which  supervisor/ 
leadership  responsibilities  constitute  new  distensions  of  job  content,  the 
second-tour  soldiers  in  all  HOS  are  responsible  for  the  performance  of 
subordinates.  The  technical  content  of  the  jobs  Is,  for  the  sx>st  part, 
similar  to  the  content  of  first-tour  jobs,  although  higher  proficiency  is 
often  expected,  and  more  difficult  tasks  are  frequently  added. 

The  content  of  the  perTormance  examples  or  incidents  gathered  for  the 
nine  MOS  allows  a  rough  estimate  of  the  relative  isportance  of  the  technical 
and  supervisory  aspects  of  the  second-tour  soldier  job.  Recall  that  the 
NCOs  and  their  supervisors  from  each  of  the  target  MOS  were  asked  in  a 
workshop  setting  to  record  behavioral  incidents  they  recalled  from  observing 
second-tour  soldiers  working  in  these  HOS;  workshop  participants  were  told 
that  the  incidents  could  refer  to  any  part  of  the  job.  We  would  expect  the 
content  of  the  incidents  gathered  in  this  manner  to  representatively  sample 
the  critical  elements  of  the  job.  More  precisely,  we  would  expect  the 
performance  incidents  elicited  in  this  way  to  reflect  what  it  takes  to  be 
effective  on  these  jobs  (rather  than,  for  example,  the  time  spent  on 
different  job  activities). 

Table  5.4  shows  the  percentage,  as  judged  by  the  project  research 
staff,  of  supervisory  performance  incidents  for  each  of  the  nine  MOS,  along 
with  the  total  percentage  of  MOS-specific  incidents  that  were  supervisory  in 
nature  across  all  nine  MOS.  In  general,  second-tour  MOS  1 IB  Infantrymen  and 
63B  Light  Wheel  Vehicle  Mechanics  seem  to  do  the  most  supervising  in  the 
technical  arena,  while  19E  Armor  Crewmen  and  88M  Motor  Transport  Operators 
have  the  least  involvement  in  supervising  soldiers. 

Comparing  Table  5.4  with  Table  5.3,  notice  that  the  decision  to  develop 
(or  not  to  develop)  MOS-specific  categories  of  supervisory  job  content  for 
each  MOS  was  not  directly  related  to  the  percentage  of  supervisory  incidents 
gathered  for  that  MOS.  Rather,  as  mentioned  previously,  MOS-specific 
supervisory  categories  were  developed  only  when  the  incidents  for  that  MOS 
reflected  aspects  of  supervision  that  were  not  tapped  by  the  Army-wide 
supervisory  dimensions. 


Job  Analysis  Interviews 

The  job  analysis  interviews  showed  that,  tor  every  muS,  the  actual 
performance  of  MOS-specific  and  common  tasks  still  requires  a  substantial 
proportion  of  the  NCO's  time;  the  estimates  ranged  from  20  to  64  percent 
across  the  MOS  and  the  job  environments.  During  the  course  of  the 
interviews,  job  experts  also  rated  the  importance  of  various  activities  and 
discussed  their  ratings:  For  every  MOS,  under  every  scenario  of  job 
environment,  the  actual  performance  of  MOS-specific  and  common  tasks  was 
considered  among  the  most  important  responsibilities  of  E5  NCOs. 


156 


Tibia  S.4 


Percentage  of  Supervisory  Performance  Incidents  From  NOS-Speciftc  Workshops 


HOS  . 

Total  Number 
of  Incidents 

Number  of 
Supervisory 
Incidents 

Percent 
Supervisory 
H03-Spec1f ic 
Incidents 

115 

Infantryman 

159 

71 

44.7% 

13B 

Cannon  Crewman 

57 

13 

22.8% 

19E 

Armor  Crewman 

236 

27 

11.4% 

31C 

Single  Channel  Radio 
Operator 

212 

49 

23.1% 

63B 

Light  Wheel  Vehicle 
Mechanic 

180 

76 

42.2% 

88M. 

Motor  Transport  Operator 

184 

31 

16.8% 

71L 

Administrative 

Specialist 

156 

36 

23.1% 

91A/B 

Medical  Specialist/ 
Medical  NCO 

89 

33 

37.1% 

95B 

Military  Specialist 

234 

73 

31.2% 

Total 

1507 

409 

27.1% 

Job  Analysis  Siawaary 


General  Findings 

The  results  of  the  several  types  of  job  analyses  may  b*  summarized  as 
follows: 

•  The  cluster  structure  generated  on  first-tour  task  content  was 
Modified  to  accommodate  the  additional  second-tour  tasks. 

Clusters  became  more  clearly  differentiated,  supervisory  clusters 
were  added,  and  clusters  of  tactical  leadership  or  administrative 
duties  were  added. 

•  The  Anny-wide  dimensions  for  first  tour  were  replicated,  and  three 
Army-wide  supervisory  dimensions  were  added  to  complete  the 
picture  of  second- tour  Army-wide  performance. 

•  For  five  of  the  HOS,  second-tour  job-specific  supervisory 
dimensions  based  on  a  modified  BARS  procedure  were  added.  Only 
two  MOS-specific  BARS  were  dropped  as  inappropriate  for  second- 
tour  soldiers. 

Results  from  both  the  task-based  and  the  critical  incident-based  work, 
as  well  as  from  the  interviews  among  job  experts,  suggest  that  indeed  the 
second-tour  soldier  job  has  performance  requirements  ir.  both  the  technical 
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and  supervisory  areas.  For  most  of  the  MOS,  roughly  one-quarter  to  one- 
third  of  the  performance  demands  are  likely  to  be  supervisory  in  nature, 
with  the  rest  involving  perforaance  of  technical  tasks. 

D  Iff  erences_Acros  s.MQS 

There  is  considerable  overlap  aaong  the  MOS.  Much  of  it  Is  by  design, 
since  the  99  coenon  tasks  for  Skill  Levels  1  and  2  are  the  responsibilities 
of  all  soldiers  in  all  jobs.  While  there  light  be  differences  in  the 
specific  circumstances  under  which  the  activities  covered  by  these  tasks 
would  be  performed  by  soldiers  in  the  different  MOS,  the  Soldier's  Manual  of 
Common  Tasks  lists  the  same  initiating  conditions,  steps,  and  standards  of 
performance  for  all  soldiers  (hence  the  tere  "common  task"). 

However,  there  are  also  substantial  differences  among  the  MOS.  Beyond 
the  similarities  found  in  the  common  tasks  and  the  Army-wide  performance 
dimensions,  second-tour  soldiers  perform  a  variety  of  MOS-specific  technical 
tasks  and  supervise  other  soldiers  on  specific  technical  tasks. 


Specific  Nature  of  the  Suoarvi sion/Leadership  Component 
Activities  of  the  Junior  HCO 

As  a  category  of  job  content,  supervision  and  leadership  represent  a 
sizeable  proportion  of  the  junior  HCO  position.  For  example,  as  judged  by 
the  previously  described  job  analysis  interview  panels,  from  35  to  80 
percent  of  the  SCO's  time  is  spent  cn  supervisory  activities.  Table  5.5 
shows  the  range  of  these  time  estimates  across  MOS  and  between  duty 
positions  within  MOS.  While  these  are  rough  estimates,  they  do  argue 
strongly  that  supervision/leadership  is  a  big  part  of  the  NCO's  job. 

Table  5.6  presents  a  breakdown  of  the  MOS-specific  and  Army-wide 
supervisory  performance  incidents.  Shown  are  the  percentages  of  the  734 
total  Army-wide  incidents  reliably  retranslated  into  the  supervisory 
performance  dimensions  in  the  Array-wide  scale  development  effort. 

Also  shown  is  a  sorting  of  the  supervisory  incidents  obtained  in  the 
MOS-specific  workshops  that  could  be  attributed  to  the  three  Army-wide 
dimensions.  The  MOS-specific  dimensions  obtained  from  the  remaining 
incidents  generated  in  the  MOS-specific  workshops  were  shown  previously 
in  Table  5.3. 

Although  the  total  percentages  of  MOS-specific  and  Army-wide  generated 
supervisory  incidents  are  reasonably  close  (27.1%  and  30.5%),  the 
distribution  of  these  incidents  among  individual  supervisory  categories  is 
very  uneven  across  the  two  sources  of  incidents.  The  majority  of  the  MOS- 
specific  supervisory  incidents  fall  in  the  Organizing,  Supervising, 
Monitoring,  and  Correcting  dimension,  whereas  supervisory  incidents  are 
somewhat  more  evenly  sprea  across  all  three  categories  in  the  Army-wide 
case. 
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Table  5.5 


Job  Analysis  Panel  Judgments  on  Percentage  of  Time  Spent  In 
Supervision  and  Laadarshlp  Activltlas 


nos 

Percent  Tine  Spent  Performing 
Supervisory Duties  - 

1  IB 

50-70 

13B 

60  -  80 

19E 

40  -  50 

31C 

55  -  75 

63B 

70 

71L 

40 

88M 

35  -  '75 

91A/B 

40 

95B 

65 

a  Range  within  MOS  is  between  garrison  and  field  situations. 


Table  5.6 

Kusber  end  Percentage  of  Performance  Incidents  by  Army-Wide 
Supervisory  Category 


HOS-Specific  Army-Wide 

Incidents  Incidents 


Cateaory 

JL- 

% 

-M _ 

Organizing,  Supervising,  Monitoring, 

. 

and  Correcting  Subordinates 

310 

2C.6 

99 

13.5 

Training  and  Developing  Subordinates 

82 

5.4 

63 

8.6 

Showing  Consideration  and  Concern 

for  Subordinates 

17 

1.1 

62 

8.4 

Total  number  of  supervisory  incidents 

409 

27.1 

224 

30.5 

Total  number  of  incidents 

1507 

724 
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The  reason  for  so  few  MOS-speciiic  incidents  in  the  Showing  Concern 
dimension  is  probably  the  Bore  gen£’ ic  nature  of  that  dimension  and  the 
instructions  to  MOS-specific  workshop  participants  to  focus  on  performance 
examples  relevant  only  to  the  target  HOS-  It  appears  that  training  is 
viewed  as  a  generic  activity  but  that  there  are  technical  components  that 
move  the  organising;  supervises,  monitoring,  and  correcting  dimension  out 
of  the  province  of  Army-wide  >erfonMt!ce  and  into  the  HOS-specific  domain. 

Dimensions  of  Supervisory  Task  Qntent 

6iven  the  sizeable  nature  of  the  supervision/leadership  component,  the 
next  step  was  to  attempt  a  more  detailed  description  of  the  content  in  terns 
of  specific  dimensions.  The  procedure  for  identifying  the  dimensions  was  as 
followr. 

An  itc*e  pool  was  created  by  first  using  project  staff  judgments  to 
identify  the  tesks  in  each  MOS  task  domain  that  represented  leadership  or 
supervision  content.  This  total  list,  sunned  over  the  nine  Batch  A  MOS,  was 
edited  for  obvious  redundance  and  then  combined  with  the  46  items  from  the 
Supervisory  Responsibilities  Quest  ionnaire/I.eader  Requirements  Survey.  This 
produced  a  total  pool  of  341  items  (tasks). 

The  pool  of  341  individual  task  item-  was  then  content  clustered  by 
each  of  12  judges  selected  from  the  Projec:  A  stuff.  Given  the  target  that 
the  nunber  of  content  clusters  should  be  between  5  and  15,  if  possible,  each 
judge  sorted  the  task  items  into  categories  and  wrote  a  brief  definition  for 
each  category  (i.e.,  dimension).  Consequently,  there  were  12  cluster 
solution-  based  on  individual  expert  judgment. 

Next,  the  degree  of  agreement  among  all  12  judges,  in  terms  of  how 
every  pair  of  items  shoti’d  e  clustered,  was  used  as  input  to  an  empirical 
cluster  analysis.  The  results  of  the  cluster  analysis  were  compared  to  the 
expert  judgment  solutions  and  a  synthe-ized  description  of  specific  content 
dimensions  was  written  by  the  project  staff.  To  say  it  another  way,  a 
pooled  solution  was  obtained  by  expert  judgment.  The  results  of  this  pooled 
sclution  are  shov/n  as  Figure  5.3. 
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A  considerable  amount  of  job  analysis  information  is  now  available  on 
which  to  base  second-tour  performance  measurement.  The  critical  incident 
analysis  yieldeo  a  portrayal  of  each  MOS  in  terms  of  its  general  and 
specific  performance  components  as  inferred  from  the  content  of  a  large  pool 
of  effective  and  ineffective  critical  incidents  generated  by  several  panels 
of  NCOs  and  officers.  A  series  of  job  analysis  interviews  provided  a  rough 
estimate  of  the  relative  importance  and  time  spent  for  technical  versus 
supervisory  activities  for  each  MO:'*.  Cluster  analyses  further  specified  the 
particular  dimensions  of  suf srviso? y/leadership  performance. 
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Activities  that  are  performed  in  advance  of  major  operations  of  a 
tactical  or  technical  nature.  That  Is,  planning  for,  getting 
ready  for,  and  developing  orders  for  various  kinds  of  team 
operations,  whether  It  he  combat,  support,  or  technical 
operations.  It  is  the  activity  that  comes  before  actual  execution 
out  in  the  field  or  work  place. 


2.  Directing/Leading  Teams 

The  tasks  in  this  category  are  concentrated  in  the  combat  and 
military  police  MOS.  They  involve  the  actual  direction  and 
execution  of  combat  and  security  team  activities.  They  occur  out 
in  the  field  and  are  heavily  dependent  on  HOS-specIflc  skills. 
Leading  reconnaissance  teams,  setting  up  offensive  and  defensive 
positions,  carrying  out  a  fire  mission,  directing  the  clearing  of 
mine  fields,  etc.  would  all  be  part  of  this  category.  They 
require  "real  time"  decision  making  under  pressure. 


3.  Monitoring/Inspecting 

This  cluster  Includes  Interactions  with  subordinates  that  seem  to 
Involve  keeping  an  operation  going  once  It  has  been  Initiated, 
such  as  checking  to  make  sure  that  everyone  is  carrying  out  their 
duties  properly,  assisting  people  to  overcome  problems,  making 
sure  everyone  has  the  right  equipment,  monitoring  or  evaluating 
the  status  of  equipment  readiness,  supply  levels,  completeness  of 
written  reports,  adequacy  of  current  operating  procedures,  etc. 
This  is  a  non-combat  or  non-crisis  set  of  activities. 


4.  Individual  Leadership 

The  content  of  the  tasks  in  this  cluster  reflects  attempts  to 
influence  the  motivation  and  goal  direction  of  subordinates  by 
means  of  goal  setting,  interpersonal  communication,  sharing 
hardships,  building  trust,  etc. 


Figure  5.3.  Supervision/Leadership  Task  Categories  Obtained  by  Synthesizing 
Expert  and  Eepirical  Cluster  Analysis  Solutions 
(page  1  of  2  pages). 


5.  Acting  as  a  Model 

This  dimension  is  not  tied  to  a  specific  task  content  but  refers 
to  the  NCO  modeling  the  correct  performance  behavior  whether  it  be 
technical  task  performance  under  adverse  conditions,  or  exhibiting 
appropriate  lilitary  bearing.  The  NCO  sets  the  example. 


6.  Coui.;?ling 

A  one-on-one  interaction  with  a  subordinate  during  which  the  NCO 
provides  support,  guidance,  assistance,  and  feedback  on  specific 
performance  or  personal  problems  that  the  soldier  might  be 
experiencing.  It  includes  counseling  on  problems  of  a 
disciplinary  nature. 


7.  Communication  With  Subordinates,  Peers,  and  Supervisors 

The  tasks  in  this  category  deal  with  composing  specific  types  of 
orders,  briefing  subordinates  on  things  that  are  happening,  and 
comnuni eating  information  up  the  line  to  superiors,  as  with  peers. 
Information  is  disseminated  in  both  written  and  oral  formats. 


3.  Training  Subordinates 

This  very  distinct  cluster  of  tasks  describes  the  day-tr-day  role 
oi  the  NCO  as  a  trainer  for  individual  subordinates.  When  such 
tasks  are  being  executed,  they  are  clearly  Identified  as 
instructional  (as  distinct  from  evaluations  or  disciplinary 
actions).  They  involve  scheduling,  planning,  and  conducting 
training. 


9.  Personnel  Administration 

Tnis  category  ic.  icade  up  of  "paperwork"  or  administrative  tasks 
that  involve  actually  doing  performance  appraisals,  making  or 
recommending  various  personnel  actions,  keeping  and  maintaining 
adequate  records,  and  following  standard  operating  procedures  for 
Army  personnel  practices. 


Figure  5.3.  Suparvlslon/Leadership  Task  Categories  Obtained  by  Synthesizing 
Expert  end  Empirical  Cluster  Analysis  Solutions 
(page  2  of  2  pages). 
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For  each  MOS,  task  selection  panels  of  job  experts  have  designated  30 
technical  (MOS-specif ic  and  coamon)  tasks  and  15  supervisory  tasks  to  be 
tested  to  evaluate  second-tour  performance.  The  technical  tasks  represent 
all  of  the  nonsupervisory  task  clusters  in  each  MOS.  Each  selected  task  has 
been  judged  in  terms  of  its  relative  criticality,  difficulty,  and  cluster 
content.  In  addition,  all  45  selected  tasks  were  rank  ordered  in  terns  of 
their  overall  importance  to  the  MOS. 

The  considerable  degree  of  convergence  across  all  methods  reinforces 
the  descriptions  of  how  job  content  differs  across  MOS,  of  how  second-tour 
positions  differ  from  first  tour,  and  of  the  supervisory/leadership  content 
of  each  MOS. 

The  procedure  for  FY88  is  to  develop  actual  performance  measures  of  the 
major  performance  components  in  each  MOS.  Given  available  resources, 
constraints  on  testing  time,  and  guidance  from  the  literature  and  previous 
Project  A  work,  a  tentative  set  of  measurement  methods  will  be  identified 
and  reviewed  by  the  project  staff  and  the  Scientific  Advisory  Group.  The 
review  will  include  considerations  of  feasibility,  cost,  estimated  construct 
validity,  and  appropriateness  for  the  job  content  identified  by  the  job 
analyses. 

For  the  methods  that  survive  the  above  review,  prototype  instruments 
will  be  pilot  tested  during  the  first  quarter  of  FY88.  After  pilot  test 
modifications,  the  revised  criterion  measures  will  be  field  tested  on  larger 
samples  during  the  second  quarter  of  FY88.  After  completion  of  the  field 
tests,  the  performance  measures  will  be  piven  their  final  revision  before 
the  second-tour  data  collection  during  tne  Longitudinal  Validation.  The 
results  of  the  second-tour  criterion  construction  efforts  will  be  reported 
in  the  Annual  Recort  for  FY88. 
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Chapter  6 

PREDICTOR  DATA  COLLECTION  FOR  LON&ITVPINAL  VALIDATION1 


During  the  fifth  year  of  Project  A  the  major  Longitudinal  Validation 
activities  with  respect  to  predictor  measurement  and  data  collection 
involved  the  following: 

(1)  Preparing  tne  predictor  battery  for  administration  to  the 
Longitudinal  Validation  (LV)  sample, 
f 2 J  Obtaining  the  ailitary  organizational  support  necessary, 
f 3 J  Designing  the  data  collection  procedures. 

(41  Hiring  and  training  the  data  collection  staff. 

(5)  Completing  most  of  the  LV  predictor  data  collection. 

In  this  chapter,  after  a  brief  explanation  of  where  the  LV  predictor  data 
collection  fits  into  the  overall  Project  A  Research  Plan,  tne  activities 
listed  above  are  described  in  wore  detail. 


THE  CONTEXT 

The  Annual  Report  for  FY  1986  described  the  Concurrent  Validation  (CV) 
data  collection  and  the  basic  analyses  of  those  data.  Those  analyses  were 
used  to  revise  the  predictor  battery,  called  the  Trial  Battery  during  CV 
administration,  for  its  administration  in  the  first  LV  phase  as  the  Experi¬ 
mental  Predictor  Battery. 

Figure  6.1  again  portrays  the  overall  data  collection  plan  for  Project 
A,  which  stipulated  that  the  Experimental  Battery  was  to  be  administered  to 
a  longitudinal  sample  consisting  of  soldiers  entering  the  service  in  FY 
86/87.  This  sample  was  to  provide  the  data  base  for  &  study  of  the  longi¬ 
tudinal  validity  of  the  Experimental  Battery,  to  build  on  and  expand  the 
validity  evidence  obtained  from  the  CV  sample. 

One  objective  of  the  LV  data  collection  was  to  administer  the  predic¬ 
tors  as  closely  as  possible  to  the  point  where  they  would  ultimately  be 
administered  operationally.  Testing  during  reception  station  processing  of 
recruits  was  chosen  as  t.he  most  feasible  method  of  obtaining  the  desired  LV 
sample.  Collecting  predictor  data  during  reception  processing  it  only  one 
step  removed  from  collecting  data  at  the  Military  Entrance  Processing 
Stations  (MEPS),  where  the  predictors  would  be  used  operationally.  Since 
all  incoming  soldiers  must  be  processed  through  one  of  eight  Reception 
Battalions  in  the  continental  United  States,  it  was  eminently  more  practical 
to  administer  the  Experimental  Predictor  Battery  in  these  battalions  rather 
than  at  the  vastly  larger  number  of  MEPS  that  are  located  throughout  the 
country. 


*The  material  in  this  chapter  was  drafted  by  Janis  S.  Houston  of  the 
^rsonnel  Decisions  Research  Institute. 


As  specified  in  the  Research  Plan,  the  Longitudinal  Validation  effort 
was  to  begin  with  adsaini strati on  of  the  Experimental  Battery  of  tests  tc 
recruits  at  the  reception  station,  and  then  would  follow  these  soldiers 
through  their  Advanced  Individual  Training  (AIT),  where  they  would  be 
administered  several  criterion  oeasures  of  performance  during  training. 
Soldiers  In  the  LV  s&epie  would  then  be  followed  into  their  first  tour, 
during  whi  h  the  first-tour  job  performance  criterion  measures  would  be 
administered.  Eventually  soldiers  who  reenlistad  would  bn  followed  Into 
their  second  tour,  where  the  second-tour  performance  measures  could  be 
administered.  This  LV  data  collection  scheme  is  summarized  schematically 
in  Figure  6.2.  The  first  phase,  administration  of  the  predictor  tests,  is 
described  in  this  chapter. 


THE  SAMPLE 


To  obtain  MOS  samples  large  enough  for  the  desired  validity  analyses, 
given  estimated  attrition  rates  over  the  period  of  the  validation  assess¬ 
ment,  each  of  the  eight  Reception  Battalions  was  asked  to  test  all  Regular 
Amy  soldiers  entering  any  one  of  the  21  MOS  listed  in  Table  6.1  for  an 
entire  calendar  year.  The  intend  was  to  test  a  total  of  approximately 
50,000  soldiers  for  the  86/87  sample,  with  the  goal  of  preserving  a  sample 
of  400-600  job  incumbents  in  each  of  the  21  MOS  for  the  LV  test  sequence. 

Although  considerably  Kire  than  50,000  Regular  Army  accessions  enter 
the  21  target  MOS  each  year,  numerous  cases  could  be  expected  in  which  all 
soldiers  in  the  target  sample  could  not  be  tested.  For  example,  on  the 
occasions  when  the  usual  3-day  (72-hour)  processing  schedule  must  be 
shortened  to  2  days,  it  would  simply  not  be  feasible  to  administer  4  hours 
of  predictors  to  these  “two-day- shippers.*  Another  Instance  Is  when  a  group 
of  soldiers  at  the  reception  station  is  sent  off-site  to  be  processed  by 
Army  Reserve  units.  Also,  when  large  numbers  of  recruits  were  being 
processed,  time  would  not  be  available  for  all  members  of  a  group  to  fake 
the  computer-administered  portions  of  the  tests.  Even  given  such  con¬ 
straints,  a  goal  of  50,000  soldiers  seemed  reasonable. 


The  start-up  dates  for  administering  the  Experimental  Battery  varied 
from  one  Reception  Battalion  to  another.  This  allowed  project  staff  members 
to  be  physically  present  during  start-up  at  each  site,  to  provide  on-the-job 
training  of  the  test  administrators,  and  to  help  resolve  any  problems  that 
arose  as  testing  actually  got  underway.  The  testing  sites  and  the  predictor 
data  collection  period  for  each  site  are  as  follows: 


Predictor  Testing  Period 


fort  Sill 
Fort  Benning 
Fort  Bliss 
Fort  Knox 
Fort  McClellan 
Fort  Cix 

Fort  Leonard  Hood 
Fort  Jackson 


20  Aug  86  •  20  Aug  87 
27  Aug  66  -  27  Aug  87 
4  Sep  86-4  Seo  87 
10  Sep  8b  -  10  Sep  87 
17  Sep  86  -  17  Sep  87 
24  Sep  86  -  24  Sep  87 
1  Oct  86  -  1  Oct  87 
19  Nov  86  -  19  Nov  87 
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PREDICTOR  BATTERY 


JOB  KNOWLEDGE  TESTS 
ARMY-WIDE  RATINGS 
(PEERS  A  SUPERVISORS) 


IMS- 

IMS 


FYM/S7 
1st  TOUR 


ADMINISTRATIVE  MEASURES 
Hands-on  tests 
JOB/TASK  KNOWLEDGE  TESTS 
AHMY-WIDfc  NATlNU* 
MOS-SPECIRC  RATINGS 


t  - 

1990-  FYM/87  SECOND-TOUR 

1991  2nd  TOUR  PERFORMANCE  MEASURES 


Figure  6.2  Data  collection  scheme  (or  Longitudinal  Validation. 
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Tabic  $.1 


Project  A  NOS  In  Longitudinal  Validation  Supl« 


Batch  A 


1 IB  Infantrywan 

138  Cannon  Crewman 

19E/M48  H60  Armor  Crewman 

19K  Ml  Armor  Crewman 

31C  Single  Channel  Radio  Operator 

638  Light-Wheel  Vehicle  Mechanic 

711  Administrative  Specialist 

88M  Motor  Transport  Operator 

91A  Medical  Specialist 

958  Military  Police 


Combat  Engineer 

MANPAOS  Crewman 

Tow/Dragon  Repairer 

Como. •'Electronics  Radio  Repairer 

Carpentry/Masonry  Specialists 

NBC  Specialist 

Ammunition  Specialist 

Utility  Helicopter  Repairer 

Unit  Supply  Specialist 

Food  Service  Specialist 

Intelligence  Analyst 


THE  EXPERIMENTAL  BATTERY 


The  Experimental  Predictor  Battery  consists  of  six  timed  cognitive 
paper-and-panci!  tests,  10  esmputer-adsinistered  tests,  and  three  untimed 
ncn-cognitive  paper-and-pencll  inventories.  A  soldier  needs  approximately  4 
hours  to  complete  the  battery,  with  the  following  breakdown  by  type  of  test: 


Timed  Tests 
Computerized  Measures 
Untimed  Inventories 


1.50  hours 
1.25  hours 
1.25  hours 


ll 


Table  6.2  shows  the  complete  array  of  tests  and  inventories  in  the 
Experimental  Battery,  the  number  of  items  in  each,  and  the  time  limit  (for 
the  timed  tests)  or’ approximate  time  to  finish  (for  the  untimed  measures). 

In  general,  the  six  cognitive  tests  assess  spatial,  wap-reading,  and 
mechanical  abilities,  and  the  10  computerized  tests  measure  various  percep¬ 
tual  and  psychomotor  abilities.  The  A8LE,  AVOICE,  and  JOB  inventories 
contain  measures  of  temperament,  biodata,  vocational  interests,  and  work 
environment  preferences.  Complete  descriptions  of  these  tests,  their  item 
content,  and  format  can  be  found  in  prior  Annual  Reports  and  other  Project  A 
reports  and  wiil  not  be  repeated  in  detail  here.  Modifications  made  for  the 
Experimental  Battery  administration  are  described  below. 
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Table  6.2 

Description  of  Tests  In  Experimental  Predictor  Battery 


Tice  Limit 

Cognitive  Paper-And-Pencil  Tests  Number  of  Itsws  jjBXSMiSll 


Reasoning  Test 
Object  Rotation  Test 
Orientation  Test 
Haze  Test 
Hap  Test 

Assembling  Objects  Test 


30 

90 

24 

24 

20 

36 


12 

7.5 
10 

5.5 
12 
13 


Computer -Admin istered  Tests 

Demographics 

Reaction  Time  1 

Reaction  Time  2 

Memory  Test 

Target  Tracking  Test  1 

Perceptual  bpeea  ana  Accuracy  Test 

Target  Tracking  Test  2 

Number  Memory  Test 

Cannon  Shoot  Test 

Target  Identification  Test 

Target  Shoot  Test 


Number  of  I  team  Apprgyimflte  IliRS. 
2  4 


15 

2 

30 

o 

36 

7 

18 

8 

ao 

6 

18 

7 

28 

10 

36 

7 

36 

4 

30 

5 

Non-Cognitive  Paper-And-Pencil 
» - 

Aiivuiicv/i  icj 

Number  of  Items 

Approximate  Time 

Assessment  of  Background  and  Life 
Experiences  (ABLE) 

199 

35 

Army  Vocational  Interest  Career 
Examination  (AV0ICE) 

182 

20 

Job  Orientation  Blank  (JOB) 

31 

5 
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As  mentioned  earlier,  project  staff  used  the  Information  obtained  from 
the  analyses  of  CV  data  to  make  the  final  revisions  to  the  predictor 
battery,  preparatory  to  its  use  in  the  Longitudinal  Validation  phase.  These 
revisions  were  not  substantial,  since  the  battery  had  already  been  through 
several  iterations  of  data  collection,  analysis,  and  revision.  The 
revisions  incorporated  into  the  Experimental  Battery  on  the  basis  of  Trial 
Battery  analyses  aay  be  sumarized  as  follows. 

Of  the  six  cognitive  tests,  only  one  had  actual  change  in  item  content. 
The  Assembling  Objects  Test  was  made  more  difficult  by  adding  four  new, 
relatively  difficult  items  and  revising  three  existing  items  to  make  them 
more  difficult;  two  minutes  were  added  to  the  time  limit  for  this  test. 
Minor  modifications  to  the  instructions,  to  simplify  and  clarify,  were  made 
for  all  six  cognitive  tests.  Finally,  a  visual  aid  was  developed  to  be  used 
while  giving  the  instructions  for  the  Orientation  Test,  since  the  CV  data 
collection  experience  suggested  that  respondents  found  these  instructions 
somewhat  difficult  to  understand. 


In  the  cemputerired  portion  of  the  predictor  battery,  minor  modified* 
tlons  also  were  made  to  the  Instructions  (displayed  on  the  screen).  Several 
changes  were  made  in  the  software,  and  several  items  on  the  Target  Identifi¬ 
cation  Test  were  revised  to  better  balance  the  item  types. 


Revisions  to  the  non-cognitive  Inventories  were  somewhat  more  exten¬ 
sive.  The  ABLE  revisions  included  deleting  10  items,  revising  16  items,  and 
using  a  separate  answer  sheet  for  responding.  For  the  AYOICE  several 
changes  were  made  in  the  scoring  procedures,  switching  existing  items  to 
scales  where  their  item-total  score  correlations  were  higher,  and  in  two 
cases  combining  two  existing  scales  into  one  scale.  A  total  of  10  items 
were  dropped  from  the  AVOICE,  16  were  added,  several  scales  were  renamed, 
and  a  separate  answer  sheet  was  prepared.  The  JOB  was  shortened  by  seven 
items  and  had  five  items  reworded,  and  all  scales  were  reconstituted  and 
rer.amed  on  the  basis  of  factor  analyses  of  the  CV  data.  A  list  of  the  final 
scales  on  all  three  nonccgnitive  Inventories  appears  as  Table  6.3. 


DATA  COLLECTION  PROCEDURES 

Initiating,  designing,  coordinating,  and  monitoring  the  LV  data 
collection  required  considerable  time  and  effort.  This  section  describes 
the  process,  from  obtaining  the  necessary  military  support,  through  hiring 
and  training  testing  site  staff,  to  actual  data  collecting/monitoring. 

Obtaining  Military  Support 

Far  in  advance  of  the  actual  data  collection,  Project  A  staff  submitted 
Troop  Support  Requests  (TSRs)  to  the  Cosanariding  General  of  the  U.S.  Training 
and  Doctrine  Command  (TRADOC).  These  requests  provided  details  on  the 
purpose  of  the  data  collection  and  the  proposed  schedule  of  events, 
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Table  6.3 

ASl£,  AVOICE,  and  JOB  Scales  In  Experimental  Battery 


~  5HSS35i -  • 

Adjustment:  Emotional  Stability 

Dependability:  Mondelinquency 

Traditional  Values 
Conscientiousness 

Achievement:  Work  Orientation 
Self-Esteem 

Surgency  (Leadership/Potency):  Ooel nance 

Energy  Level 

Agreeableness/LIkablllty:  Cooperativeness 
Locus  of  Control:  Internal  Control 
Physical  Condition:  Physical  Condition 

Response  Validity  Scales:  Unlikely  Virtues  (Social  Desirability) 

Self-Knowledge 
Mon-Random  Response 
Poor  Iapresslon 


A  YOKE  Sea  lit 

Realistic:  Mechanics 

Heavy  Construction 
Electronics 

Electronic  Communication 

Drafting 

Law  Enforcement 

Fire  Protection 

Audiographics 

Rugged  Individualism 

Firearms  Enthusiast 

Combat 

Vehicle  Operator 

Conventional :  Cler ical/Administrati ve 
WsrehO'j?  i,,g/Ship,;if*'' 

Food  Service-Professional 
Food  Service-  -Emp  ioyee 

Social  &  Enterprising:  Leadership/Suidance 

Investigative:  Medical  Service 
Mathematics 
Science/Chemical 
Computers 

Artistic:  Aesthetics 


Job  Pride 

Job  Security/Comfort 
Serving  Others 
Job  Autonomy 
Job  Routine 
Ambition 
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location,  number  of  hours  required  of  each  soldier  tested,  and  complete 
requirements  for  personnel,  classrooms,  and  equipment. 

After  the  T5Rs  were  submitted,  senior  project  staff  met  with  the  Chief 
Executive  Officers  (four-star  generals)  of  the  organisations  providing 
support.  Numerous  briefings  were  conducted  at  various  points  down  the  chain 
of  conuvand,  culminating  in  several  meetings  with  the  Points  of  Contact  (POC) 
at  each  of  the  eight  Reception  Battalion  sites,  several  months  prior  to  data 
collection  at  that  site.  From  this  point  until  testing  began,  coordination 
was  taken  over  by  the  POC,  who  was  responsible  for  providing  the  required 
troops  to  be  tested,  classrooms/offices,  and  necessary  equipment  (e.g., 
storage  cabinets  with  locks,  phones). 

The  two  main  challenges  in  preparing  each  site  for  testing  were  (a) 
fittina  4  hours  of  testing  into  an  already  demanding  72-hour  processing 
schedule,  and  (b)  obtaining  adequate  space  for  testinp,  that  met  good 
testing  standards,  every  day  for  a  full  year.  The  solutions  to  these 
problems  were  as  many  and  varied  as  the  number  of  testing  sites.  They 
ranged  (a)  from  administering  the  predictors  in  a  single  4-hour  block  to 
administering  them  in  three  separate  sessions  over  2  or  3  days,  and  (b)  from 
redesigning/rewiring  World  War  II  barracks  to  creating  new  rooms  to  accom¬ 
modate  the  computer’ testing.  Eventually,  all  eight  reception  sites  had 
workable  testing  arrangements. 


Hiring  and  Training  Test  Site  Personnal 


lo  nanaie  data 
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trained  to  collect  the  IV  data,  rather  than  sending  project  personnel  to 
each  site  as  was  done  for  the  CY  data  collection  (which  was  much  shorter 
duration). 


and 

in 


A  Test  Site  Manager  (TSM)  was  hired  to  be  in  charge  of  each  site.  This 
manaqer  was  supported  by  from  one  to  as  many  as  eight  Test  Administrators 
(TAs)  per  site,  depending  on  the  testing  schedule  and  volume.  Applications 
were  taken  by  mail  for  both  positions,  and  all  initial  interviewing  and 
hiring  was  done  on  site  by  experienced  Project  A  staff. 

The  staff  prepared  test  administration  manuals,  with  data  collection 
procedures  described  in  minute  detail.  These  manuals  were  used  as  the  basis 
for  a  1-week  training  course,  conducted  at  each  site  for  the  newly  hired 
test  site  personnel.  Two  experienced  project  staff  members  conducted  the 
training  sessions  at  each  site.  They  were  also  present  for  the  first  2  or  3 
days  of  each  site's  predictor  testing,  to  ensure  that  all  was  going  smoothly 
and  to  provide  additional  on-the-job  training. 

Data  Collactinq/Honitorlno 

As  previously  mentioned,  the  testing  schedule  varied  from  site  to  site. 
It  was  often  necessary  to  schedule  different  portions  of  the  Experimental 
Battery  for  different  days.  However,  regardless  of  when  each  subset  of  the 
predictors  was  administered,  the  precise  directions  for  administering  each 
predictor  did  not  vary.  To  ensure  standardization  for  tests  across  sites 
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and  situations,  scripts  were  prepared  for  each  test  or  Inventory  (including 
a  script  to  precede  the  self-adainistration  computer  measures)  and  test  site 
personnel  were  trained  in  their  use.  Extensive  training  and  practice  was 
also  given  in  handling  questions  from  respondents. 

Another  feature  of  the  data  collection  process  that  helped  maintain 
standardization  of  tests  across  sites  was  the  weekly  phone  report  that  TSMs 
were  required  to  make.  Each  week,  at  a  pre-arranged  tine,  the  TSM  called 
the  Project  A  staff  person  in  charge  of  the  data  collection  and  reported  the 
number  of  soldiers  tested  the  previous  week,  discussed  any  questions  or 
problems  he  or  she  had,  and  received  relevant  news  or  instructions. 

In  addition  to  the  weekly  phone  reports,  the  site  managers  were 
required  to  submit  monthly  written  reports  of  their  testing  progress,  along 
with  documentation  of  any  problems  that  had  occurred  or  events  that  may  have 
had  an  impact  on  test  resuits  (e.g.,  interruptions  during  testing  sessions, 
a  soldier  falling  asleep  during  a  timed  test). 

Monitoring  site  visits  were  scheduled  so  that  Project  A  and/or  ARI 
staff  visited  each  site  from  one  to  three  times  throughout  the  year's 
testing.  The  purpose  of  these  visits  was  to  monitor  the  test  administra¬ 
tion,  providing  feedback  where  appropriate,  go  over  any  questions  or 
unresolved  problems,  and  generally  try  to  reinforce  esprit  de  corps. 

While  the  LV  predictor  data  collection  at  the  reception  stations  has 
certainly  not  been  devoid  of  problems,  the  cooperation  from  military 
personnel  on  post  has  generally  bean  very  good,  and  the  competence  of  the 
Project  A  site  personnel  has  been  comnendable. 


TESTING  COMPLETED  BY  END  OF  FY87 


Table  6.4  presents  the  number  of  soldiers  for  whom  paper-and-pencil 
predictor  data  had  been  obtained  as  of  1  October  1987,  by  site  and  by  MOS. 

A  total  of  47,896  incoming  soldiers  had  been  tested  by  that  date.  This 
number  represents  the  final  count  at  all  sites  except  Fort  Jackson,  where 
the  predictor  testinq  schedule  continued  through  19  November  1987.  All 
other  sites  had  completed  their  administration  of  the  Experimental  Predictor 
Battery  and  their  data  (paper-and-pencil  testing)  are  included  in  Table  6.4. 


The  total  number  of  soldiers  from  whom  computer-administered  predictor 
data  were  obtained  is  considerably  smaller  than  the  total  shown  In  Table  6.4 
for  paper-and-pencil  predictors.  By  1  October  1987,  approximately  38,000 
soldiers  had  completed  all  computer  predictors.  The  primary  reason  for  this 
discrepancy  is  that  only  about  30  percent  of  the  14,000-plus  soldiers  tested 
at  Fort  Banning  could  be  scheduled  for  the  computer-administered  portion  of 
the  predictor  battery.  However,  since  Fort  Benning  processes  only  Infantry¬ 
men  (MOS  11B),  and  in  very  large  numbers,  the  “missing  data"  on  the  computer 
tests  is  not  a  serious  problem;  complete  predictor  data  are  available  for 
approximately  4,500  Infantrymen. 
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Table  6.4 

LV  Predictor  Saicple  as  of  1  Ocfobar  1987 
(Papar-and-PancIl  Predictors  Only) 


LV  Reception  Site 


Fort 

Fort 

Fort 

Fort 

Fort 

Fort 

Fort 

Fort  Leonard 

80S. 

Bepflinfl 

£1121 

m 

Jto 

flgCjfeUand 

iUl 

toad 

Iaui 

1  IB 

14182 

3 

l 

i 

14187 

12B 

3 

2 

3 

2110 

2118 

13B 

4 

3 

1 

2 

5070 

5080 

16S 

681 

23 

10 

6 

12 

771 

19E 

580 

580 

19K 

1 

1843 

1844 

27E 

5 

4 

7 

69 

50 

135 

29E 

7 

190 

12 

16 

' 

16 

241 

31C 

109 

651 

76 

28 

30 

894 

51B 

27 

17 

9 

2 

401 

456 

54E 

5 

412 

6 

501 

11 

935 

55B 

46 

58 

33 

215 

124 

476 

63B 

721 

748 

119 

39 

409 

2036 

64C 

844 

104 

27 

60 

477 

1512 

67N 

28 

2 

227 

63 

14 

334 

71L 

314 

1438 

34 

146 

2 

1934 

76Y 

238 

1504 

191 

127 

432 

2492 

91A 

1141 

651 

961 

453 

177 

597 

3980 

94B 

857 

1112 

213 

228 

958 

3368 

95B 

10 

1 

8 

4181 

6 

4206 

96B 

m 

_£i 

_ i 

_ 317 

Total  14182  1822  4144  7272  3849 


5858 


5070 


5689  47896 


NEXT  STEPS  IN  LONGITUDINAL  VALIDATION 

As  a  i  October  1987,  the  only  site  still  collecting  LV  predictor  data 
was  Fort  Jackson.  Testing  was  scheduled  to  be  completed  at  this  site  19 
November  1987. 

The  work  that  remains  to  be  done  with  regard  to  LV  predictors  is  to 
finish  collecting  and  processing  these  data,  merge  the  various  predictor 
data  files  (paper-and-pencll  and  computer-administered),  and  edit  the  data 
in  preparation  for  scoring  and  analysis. 

During  FY88  soldiers  in  the  target  HOS  who  took  the  predictor  tests 
will  be  followed  into  their  training  assignments.  Data  from  training 
performance  measures  will  be  collected  as  the  soldiers  complete  AIT  or  one- 
station  unit  training  (OSUT).  Subsequently  a  sample  from  each  MOS  will  be 
tested  on  their  performance  on  the  job. 
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Chapter  7 
THE  FUTURE 


The  principal  focus  of  activity  for  Project  A  during  FY83  will  be  on 
preparing  for  and  executing  the  criterion  data  collection  for  the 
Longitudinal  Validation.  The  sample  of  MOS  that  are  now  included  as  part  of 
the  project  were  listed  in  Table  6.1. 

The  troop  support  requests  for  the  LV  sample  followup  for  these  21  MOS 
ask  for  a  total  of  15.00C  first-tour  job  incumbents,  all  of  whom  were  tested 
with  the  Experimental  Predictor  Battery  when  they  entered  the  Army  during 
FY87.  The  LV  sample  will  also  include  as  many  as  possible  of  those 
individuals  from  the  Concurrent  Validation  sample  (83/84  cohort)  who 
reenlisted  and  can  be  assessed  on  a  battery  of  performance  measures 
appropriate  for  second-tour  personnel. 

The  major  steps  to  be  accomplished  in  this  phase  of  the  project  during 
FY88  are  as  follxs: 

a  The  administration  of  the  Experimental  Predictor  Battery  to  the 
sample  of  new  accessions  in  the  Longitudinal  Validation  sample 
must  be  completed.  Data  remain  to  be  collected  at  only  one  of  the 
eight  reception  stations. 

e  The  administration  of  the  training  performance  measures  (i.e.,  the 
rating  scales  and  knowledge  tests)  to  the  individuals  in  the 
sample  at  the  end  of  their  Advanced  Individual  Training  roust  be 
completed. 

•  The  job  performance  criterion  measures  that  were  used  to  evaluate 
first-tour  job  incumbents  in  the  Concurrent  Validation  sample  must 
be  revised,  updated,  reviewed,  and  made  ready  for  administration 
to  first-tour  job  Incumbents  in  the  LV  sample. 

•  Job  performance  criterion  measures  for  the  second-tour  positions, 
based  on  the  job  analyses  reported  in  Chapter  5,  muii  be 
developed,  field  tested,  and  prepared  for  administration  to 
members  of  the  CV  sample  who  have  reenlisted.  Special  attention 
must  be  given  to  measures  of  the  supervisory/leadership  components 
of  NCO  positions. 

•  Data  collection  teams  for  the  LV  criterion  data  collection  must  be 
identified  and  trained.  The  specific  data  collection  procedures 
for  administering  criterion  measures  to  the  LV  sample  (first  tour) 
and  the  CV  followup  sample  (second  tour)  are  to  be  developed. 

•  First-tour  and  second-tour  job  performance  criterion  measures  must 
be  administered  to  sample  members  of  the  86/87  olort  and  83/84 
cohort  respectively. 
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While  the  above  activities  will  occupy  the  Project  A  staff  during  FY88, 
much  remains  to  be  done  to  accomplish  the  objectives  of  the  LV  data 
collection,  as  well  as  longer  range  goals.  Extensive  data  analyses  of 
second-tour  data  will  be  needed  to  develop  basic  job  performance  scores  for 
NCO  positions.  The  portrayal  of  first-tour  performance  developed  during  the 
CV  sample  analysis  needs  to  be  put  to  a  confirmatory  test  with  data  from  the 
LV  sample.  The  covariance  structure  of  the  Experimental  Predictor  Battery 
should  be  modeled,  and  the  validation  results  obtained  with  the  CV  sample 
should  be  cross-validated  with  the  LV  sample. 

Given  a  reasonable  portrayal  of  the  covariance  structures  of  the 
predictors,  training  criteria,  first-tour  peiformance  criteria,  and  NCO 
performance  criteria,  a  great  deal  of  analyses  will  be  needed  to  portray  the 
effects  of  alternative  strategies  for  selection  and  classification.  These 
analyses  would  include  the  optimal  weightinq/keying  of  predictor  data  and 
sensitivity  analyses  under  different  goals  (e.g.,  maximize  aggregate 
performance  vs.  minimize  attrition)  and  varying  constraints  (e.g.,  different 
testing  times). 

Consequently,  while  much  of  the  work  planned  under  Project  A  has 
already  been  accomplished,  more  remains  to  be  done  to  make  the  most 
effective  use  of  the  information  that  has  been  and  is  being  assembled.  A 
great  deal  of  excitement  lies  ahead. 
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COMPARISON  OF  VALIDITY  AMD  SUBGROUP  STATISTICS 
FOR  CURRENT  AND  PROPOSED  APTITUDE  AREA  COMPOSITES, 
BASED  OM  SQT  CRITERIA 
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AA  Coqposlto 

AANEW  -  Proposed  ASVA6  Aptitude  Area  Composite 
AAOID  -  Current  ASVAB  Aptitude  Area  Composite 

AACUT  -  Minimum  qualifying  score  on  the  current  AA  Composite 
(relative  to  reference  mean  of  100  and  SO  of  20) 

HTOT  -  Total  number  of  SQT  examinees  analyzed 


Estimated  Validity 

VNEW  -  Validity  of  the  proposed  AA  composite,  corrected  for  range 
restriction 

VOLD  -  Validity  of  the  current  AA  composite,  corrected  for  range 
restriction 

VCH  -  Increase  in  validity.  A  positive  value  means  the  proposed 
composite  is  more  valid  that  the  current  one. 
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For  the  proposed  composite,  the  difference  between  the 
predicted  performance  based  on  the  coamon  regression  expressed 
in  standard  deviation  units.  A  positive  value  means  that 
predicted  performance  is  higher  using  the  common  regression 
line  (l.e.,  blacks  are  overpredicted  using  the  common 
regression  equation). 


For  the  current  composite,  the  same  difference  as  for  BPDN. 

Difference  in  magnitude  of  the  black  prediction  errors  based  on 
the  current  composite  and  on  the  proposed  composite.  A 
positive  value  means  that  the  proposed  new  AA  composite 
predicts  more  accurately  for  blacks  than  the  current  composite. 


Number  of  female  examinees  analyzed 
Equivalent  of  BPDN  for  female  differences 
Equivalent  of  BPDO  for  female  differences 
Equivalent  of  PBDCH  for  female  differences 
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IhSTRUCTIOKS  AMD  MATERIALS  USED  TO  COLLECT 
WEIGHTING  JUDGMENTS  FOR  INFANTRYMEN  (NOS  11B) 
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JUDGING  THE  IMPORTANCE  OF  PERFORMANCE  FACTORS  IN  ARRIVING 

AT  TOTAL  SCORES 


Background 

A  number  of  different  kinds  of  performance  factors  are  being  considered 
by  Project  A  to  assess  the  effectiveness  of  first-tour  enlisted  personnel. 
These  various  performance  factors  must  be  combined  into  one  overall  measure 
of  MGS  performance.  This  overall  measure  should  be  the  best  that  can  be 
obtained  given  the  available  component  performance  factors.  The  overall 
treasure  will  be  used  as  the  performance  measure  against  which  the  ASVAB  and 
other  predictor  performance  factors  will  be  validated.  To  obtain  the  best 
overall  measure  for  each  HOS  in  our  sample,  Project  A  staff  will  be  asking 
knowledgeable  officers  and  KCOs  to  assign  weights  tc  the  various  performance 
factors  in  a  Banner  that  reflects  the  relative  importance  that  the  component 
performance  factors  should  have  in  forming  an  overall  measure  for  the  HOS. 

Today  we  would  like  to  get  your  judgments  about  the  relative  weights 
that  the  factors  should  receive  in  deriving  an  overall  performance  measure 
for  first-tour  Infantryman  (11B).  The  performance  factors  are 

Task  Proficiency:  HOS  specific  technical  skills--This  performance 
factor  represents  the  proficiency  with  which  the  soldier  performs  the 
tasks- which  are  'central*  to  MOS  11B.  The  tasks  represent  the  core  of 
the  job  and  they  are  the  primary  definers  of  the  HOS.  For  example,  ihc- 
first  tour  Infantryman  engages  enemy  target  with  hand  grenades; 
installs  and  fires/recovers  an  M18A1  claymore  mine;  selects  hasty 
firing  positions  in  urban  terrain;  zeros  an  Afi/PVS-4  to  an  H16A1  rifle; 
and  uses  weapons  and  other  equipment  in  offensive  and  defensive  combat 
operations. 

This  performance  factor  does  not  include  the  individuals  willingness 
to  perform  the  task  or  the  degree  to  which  the  individual  can 
coordinate  his  efforts  with  others.  It  refers  to  how  well  the 
individual  can  execute  the  core  technical  tasks  the  job  requires,  given 

.•  i  i  ; _ «...  .. 

a  mi  mynciJ  lv  uws  jv  * 

Task  Proficiency;  General  soldiering  skills— In  addition  to  the  core 
technical  content  specific  to  an  MOS,  individuals  in  every  MOS  are  also 
responsible  for  being  able  to  perform  a  variety  of  general  soldiering 
tasks--for  example,  determines  grid  coordinates  on  military  maps,  puts 
on,  wears  and  removes  M17  series  protective  mask  with  hood,  determines 
a  magnetic  azimuth  using  a  compass,  collects/ reports  information-- 
SALUTE,  recognizes  and  identifies  friendly  and  threat  aircraft. 
Performance  or:  this  factor  represents  overall  proficiency  on  these 
general  soldiering  tasks.  Again,  it  refers  to  how  well  the  individual 
can  execute  general  soldiering  tasks,  given  a  willingness  to  do  so. 


Exercise  of  Leadership.  Effort,  and  Self  Development-  Is  performance 
factor  reflects  the  degree  to  which  the  Tndividual  exerts  effort  oyer 
the  full  range  of  job  tasks  perseveres  under  adverse  or  dangerous 
conditions,  and  demonstrates  leadership  and  support  toward  peers.  That 
Is,  can  the  individual  be  counted  on  to  carry  out  assigned  tasks,  ever 
under  adverse  conditions,  to  exercise  good  judgment,  and  to  be 
generally  dependable  and  proficient.  While  appropriate  knowledges  and 
skills  are  necessary  for  successful  performance,  this  factor  is  only 
meant  to  reflect  the  Individual's  willingness  to  do  the  job  required 
ar.d  to  be  cooperative  and  supportive  with  other  soldiers. 

Maintaining  Personal  Discioline—This  performance  factor  reflects  the 
degree  to  which  the  individual  adheres  to  Army  regulations  and 
traditions,  exercises  personal  self  control,  demonstrates  integrity  in 
day  to  day  behavior,  and  does  not  create  disciplinary  problems.  People 
who  rank  nigh  on  this  factor  show  a  commitment  to  high  standards  of 
personal  conduct. 

Military  Bearino/Appearance  and  Physical  Fltnes^-This  performance 
factor  represents  the  degree  to  which  the  individual  maintains  an 
appropriate  military  appearance  and  bearing  and  stays  in  good  physical 
condition. 


Please  assume  that  a  total  score  will  be  derived  for  each  soldier  from 
the  separate  scores  obtsined  from  each  of  these  factors.  These  total  scores 
will  be  our  best  estimate  of  the  overall  effectiveness  of  the  troops  whose 
performance  will  be  measured.  We  need  the  assistance  of  experienced  Army 
personnel  in  determining  how  much  weight  should  be  given  each  factor  in 
arriving  at  the  total  effectiveness  scores. 


Purpose 

The  purpose  of  this  workshop  is  to  obtain  the  weights  to  be  assigned 
each  of  the  performance  factors.  Two  methods  of  assigning  weights  will  be 
used.  The  methods  differ  in  the  kinds  cf  judgments  you  will  be  required  to 
make: 

Method  A:  You  will  be  asked  to  rank  order  the  performance  factors  and  then 
assign  weights  to  them,  assuming  that  the  top  ranked  factor  has  a 
weight  of  100. 

Method  B:  You  will  be  given  performance  profiles  on  10  sets  of  15  soldiers 
each  and  asked  to  rank  order  them.  (The  profiles  will  give  the 
scores  of  the  soldiers  on  two  of  the  five  performance  factors  at  a 
time. ) 
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Assumptions  for  Both  Methods 


(1)  The  type  of  soldiers  for  whoa  performance  factor  weights  are  being 
derived  is  first  tour  Infantryman  (11B). 

(2)  As  the  weights  you  assign  may  be  a  function  of  the  particular 
context  in  which  the  soldiers'  performance  is  being  evaluated, 
please  assume  the  following  military  situation  prevails: 

The  world  is  In  a  period  of  heightened  tensions.  There 
is  an  increasing  probability  that  hostilities  will  break 
out  In  Europe,  Asia,  the  Caribbean,  Latin  America,  and 
Africa.  The  Army's  mission  is  to  support  U.S.  treaty 
obligations  and  to  help  defend  the  borders  of  allied 
and  friendly  nations.  Some  of  the  potential  enemies 
have  nuclear  and  chemical  capability.  Air  parity  does 
exist  between  allied  forces  and  potential  hostile  nations. 
U.S.  Army  training  and  other  preparatory  activities  have 
been  substantially  increased.  Host  combat  and  associated 
support  units  are  participating  in  frequent  field  exercises. 
Host  units  are  being  actively  resupplied. 

(3)  Performance  factor  scores  are  available  only  on  the  factors  given. 
Although  there  may  he  other  factors  that  comprise  overall 
performance,  no  scores  are  available  for  them  at  this  time. 
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DIRECTIONS  FOR  METHOD  A 


Under  this  weighting  Method,  the  procedure  for  assigning  weights  to  the 
performance  factors  Is  as  follows: 

1.  Rank  order  the  set  of  performance  factors  to  be  weighted  by 
assigning  a  *1*  to  the  most  Important,  a  *2*  to  the  next  most 
Important,  etc.  Please  refer  to  the  ■PERFORMANCE  FACTORS  FOR 
MOS  118"  handout  for  a  complete  description  of  the  5  performance 
factors. 

2.  After  you  have  recorded  the  rank  orders  on  the  weighting  sheet, 
assign  100  points  to  the  factor  you  ranked  as  most  Important. 

Then  ask  yourself,  *If  I'm  assigning  100  points  to  this 
performance  factor,  how  many  points  should  I  assign  to  the 
next  most  Important  one.*  If,  for  example,  you  think  that  the 
second  most  Important  one  should  receive  half  the  weight  of  the 
first,  assign  It  50  points.  Continue  assigning  points  In  this 
manner  until  all  the  factors  have  been  weighted. 

3.  In  assigning  the  points,  please  keep  In  mind  that  the  points 
represent  how  many  times  more  (or  less)  Important  one  performance 
factor  Is  than  another.  For  example.  If  you  assign  30  points  to 
one  factor  and  5  points  to  another,  that  means  that  you  believe 
that  the  30-point  factor  should  receive  6  times  the  weight  In  the 
total  score  as  the  5-point  factor. 

4.  If  you  feel  that  two  or  more  factors  should  be  weighted  equally, 
you  may  assign  them  equal  weights.  For  example,  If  you  feel  that 
the  factors  ranked  first  and  second  are  really  tied  In  Importance, 
then  you  can  assign  them  both  100  points. 

5.  If  you  believe  that  a  particular  performance  factor  should  not  be 
used  at  all  In  arriving  at  the  total  score,  you  should  assign  it 
zero  points. 

6.  When  you  are  finished  assigning  points  to  all  performance  factors, 
please  make  sure  that  they  are  in  the  "right*  ratio  to  one 
another.  That  Is,  the  points  assigned  to  all  factors  are  In 
correct  proportion  to  one  another. 

Thank  you  for  your  cooperation. 


D-5 


■  Name _ Workshop _ 

MOS  11B  Performance  Factor  Weighting  Sheet 

Rank 

Performance  Factor*  Order  Wf iqht 

1.  Task  proficiency  —  HOS  specific  technical  skills.  _  _ 

2.  Task  proficiency  —  general  soldiering  skills.  _ _  _ 

3.  Exercise  of  leadership,  effort,  and  self  development.  _  _ 

4.  Maintaining  personal  discipline.  _  _ 

5.  Military  bearing/appearance  and  physical  fitneii.  _ _  _ 

*  Please  refer  to  the  ’PERFORMANCE  FACTORS  FOR  HOS  118"  handout  for  a 
complete  description  of  the  5  performance  factors. 
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PERFORMANCE  FACTORS  FOR  NOS  UB 


1)  Task  Proficiency:  MOS  specific  technical. skills 

This  performance  factor  represents  the  proficiency  with  which  the  soldier 
performs  the  tasks  which  are  "central*  to  MOS  11B.  The  tasks  represent  the 
core  of  the  job  and  they  are  the  primary  definers  of  the  HOS.  For  example, 
the  first  tour  Infantryman  engages  enemy  target  with  hand  grenades;  installs 
and  fires/recovers  an  M18A1  claymore  mine;  selects  hasty  firing  positions  in 
urban  terrain;  zeros  an  AM/PVS-4  to  an  H16A1  rifle;  and  uses  weapons  and 
other  equipment  in  offensive  and  defensive  combat  operations. 

This  performance  factor  does  not  Include  the  individual's  willingness  to 
perform  the  task  or  the  degree  to  which  the  individual  can  coordinate  his 
efforts  with  others.  It  refers  to  how  well  the  individual  can  execute  the 
core  technical  tasks  the  job  requires,  given  a  willingness  to  do  so. 

2)  Task  Proficiency:  General  soldiering  skills 

In  addition  to  the  core  technical  content  specific  to  an  MOS,  Individuals  in 
every  MOS  are  also  responsible  for  being  able  to  perform  a  variety  of 
general  soldiering  tasks— for  example,  determines  grid  coordinates  on 
military  maps,  puts  on,  wears  and  removes  MI7  series  protective  mask  with 
hood,  determines  a  magnetic  azimuth  using  a  compass,  collects/  reports 
information  -  SALUTE,  recognizes  and  Identifies  friendly  and  threat 
aircraft.  Performance  on  this  factor  represents  overall  proficiency  on 
these  general  soldiering  tasks.  Again,  it  refers  to  how  well  the  individual 
can  execute  general  soldiering  tasks,  given  a  willingness  to  do  so. 

3)  Exen;.!?*  9f  .L«ftr$hip.«.Eff9rti  Self  PsvglQBmsnl 

This  performance  factor  reflects  the  degree  to  which  the  individual  exerts 
effort  over  the  full  range  of  job  tasks,  perseveres  under  adverse  or 
dangerous  conditions,  and  demonstrates  leadership  and  support  toward  peers. 
That  Is,  can  the  individual  be  counted  on  to  carry  out  assigned  tasks,  even 
under  adverse  conditions,  to  exercise  good  judgment,  and  to  be  generally 
dependable  and  proficient.  While  appropriate  knowledges  and  skills  are 
necessary  for  successful  performance,  this  factor  is  only  meant  to  reflect 
the  individual's  willingness  to  do  the  job  required  and  to  be  cooperative 
and  supportive  with  other  soldiers. 

4)  Maintaining  Personal  Discipline 

This  performance  factor  reflects  the  degree  to  which  the  Individual  adheres 
to  Army  regulations  and  traditions,  exercises  personal  self  control,  demon¬ 
strates  integrity  in  day-to-day  behavior,  and  does  not  create  disciplinary 
problems.  People  who  rank  high  on  this  factor  show  a  commitment  to  high 
standards  of  personal  conduct. 

5)  Military  Rearino/Appcarance  and  Physical  Fitness 

This  performance  factor  represents  the  degree  to  which  the  individual  main¬ 
tains  an  appropriate  military  appearance  and  bearing  and  stays  in  good 
physical  condition. 
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DIRECTIONS  FOR  METHOD  B 


Under  this  Method,  judgments  of  the  overall  performance  scores  for  10 
sets  of  Infantrymen  will  be  obtained.  Each  set  will  contain  15  Infantrymen. 
The  performance  scores  of  each  of  the  15  first  tour  Infantrymen  have  been 
recorded  on  2  performance  factor  scales.  (A  different  pair  of  performance 
factor  scales  are  provided  for  each  of  the  10  sets).  For  each  scale  there 
is  a  description  of  high,  medium  and  low  levels  of  performance.  Each  of  the 
15  soldiers  Is  rated  on  a  7-point  scale  that  ranges  from  the  lowest  level  of 
performance  to  the  highest.  Please  refer  to  the  "PERFORMANCE  FACTORS  FOR 
MGS  11B"  handout  for  a  complete  description  of  the  5  performance  factors. 
Also,  please  review  the  assumptions  given  in  the  General  Instructions. 

1.  Rank  the  15  Infantrymen  In  the  first  set  in  order  of  their  overall 
performance.  Give  the  “best"  soldier  a  rank  of  "1",  the  second  best 
soldier  a  rank  of  "2*  and  so  on.  Make  comparisons  between  the  soldiers 
on  the  basis  of  their  overall  performance  as  Infantrymen;  do  not 
consider  how  they  might  be  used  in  other  capacities. 

2.  When  you  are  finished,  please  go  over  the  rank  order  carefully  making 
sure  that,  in  your  judgment,  the  ranks  reflect  the  relative  overall 
performance  of  the  soldiers.  Feel  ft  jc  to  change  any  ranks. 

3.  When  satisfied  with  you*"  rank  ordering,  proceed  to  the  next  set  of  15 
Infantrymen. 

Thank  you  for  your  cooperation. 


(A  sample  sheet  for  one  pair  of  performance  factor  scales  follows.) 
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Name 


Sheet  No.  1 


NOS  118  OVERALL  PERFORMANCE  SCORE  SHEET 


Performance  Level 


Task  Proficiency—  Task  Proficiency- 

Soldier  NOS  Specific  General  Rank 

Ho.  Technical  Skills  Soldiering  Skills  Order 


1  6 

2  S 

3  2 

4  5 

5  2 

6  6 

7  4 

B  1 

9  S 

10  7 

11  3 

12  3 

13  4 

14  4 

15  3 


2 

5 

6 
3 

3 
5 
7 

4 
f> 

4 

5 

3 

4 
1 
2 


Performance  Scales: 


TASK  PROFICIENCY— NOS  SPECIFIC  TECHNICAL  SKILLS 


Does  not  display  the 
knowledge/skill  required 
to  perform  many  core 
technical  skills. 

Displays  the  knowledge/ 
skill  required  to  perform 
most  core  technical  tasks 
properly,  but  may  need 
help  for  harder  tasks. 

Displays  the  knowledge/ 
skill  to  perform  all  core 
technical  tasks  properly. 

i  2 

3  4  5 

5 

7  1 

TASK  PROFICIENCY— GENERAL  SOLDIERING  SKILLS 


Does  not  display  the 
knowl edge/sk i 1 1  required 
to  perform  many  general 
soldiering  tasks. 

Displays  the  knowledge/ 
skill  required  to  perform 
most  general  soldiering 
tasks,  but  may  need  help 
for  harder  tasks. 

Displays  the  knowledge/ 
skill  to  perform  all 
general  soldiering 
ski  11s. 

1  2  3  4  5  6  7 
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Name 


Sheet  No.  2 


MOS  1 IB  OVERALL  PERFORMANCE  SCORE  SHEET 


Soldier 

No. 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

n 

12 

13 

14 

15 


Performance  Level 


Military  Bearing/ 
Appearance  and 
Physical  Fitness 

6 

5 

2 

5 

6 
4 
1 


Exercise  of  Leader 
ship.  Effort  and 
Self  Development 

2 

5 

6 
3 

3 

5 
7 

4 

6 

4 

5 

3 

4 
1 


Rank 

Order 


Performance  Scales: 


MILITARY  BEAR I NS/ APPEARANCE  AND  PHYSICAL  FITNESS 


Maintains  self  In  poor 
physical  condition.  Falls 
to  meet  military  standards 
for  dress  or  personal 
hygiene. 

Meets  Army  standards  of 
physical  fitness.  Dresses 
neatly  and  meets  Army 
standards  of  personal 
hygiene. 

_  -  - 

Exceeds  Army  standards 
and  expectations  set  for 
physical  fitness.  Main¬ 
tains  excellent  personal 
hygiene  and  proper 
appearance. 

1  2  3  4  5  6  7 

EXERCISE  OF  LEADERSHIP.  EFFORT  AMD  SELF  DEVELOPMENT 


Falls  to  take  charge  when 
leadership  Is  required  In 
unit.  Provides  little  or 
no  assistance  to  other 
unit  members.  Seldom 
exerts  effort  In  accom¬ 
plishing  many  job  assign¬ 
ments  and  tasks.  Gives 
up  easily  under  adverse 
conditions. 

Performs  satisfactorily  In 
leadership  situations 
where  what  Is  expected  Is 
well  known.  When  asked, 
gives  help  and  support  to 
fellow  soldiers.  Usually 
exerts  effort  to  perform 
most  job  assignments  and 
tasks. 

Takes  charge  when  neces¬ 
sary  to  lead  unit;  leads 
the  squad  to  outstanding 
performance.  Does 
everything  possible  to 
assist  other  soldiers. 
Always  exerts  consider¬ 
able  effort  in  performing 
all  job  assignments  and 
tasks. 

1  2  3  4  5  6  7 

D-10 


Nam* 


Sheet  No.  3 


MOS  11B  OVERALL  PERFORMANCE  SCORE  SHEET 


Performance  Level 

Maintaining 

Task  Proficiency— 

Soldier 

Personal 

MOS  Specific 

Rank 

No. 

Discipline 

Technical  Skills 

Order 

1 

6 

2 

2 

5 

5 

3 

2 

6 

4 

5 

3 

5 

2 

3 

6 

6 

5 

7 

4 

7 

B 

1 

4 

9 

5 

6 

10 

7 

4 

11 

3 

5 

12 

3 

'  3 

13 

4  . 

4 

14 

4 

1 

15 

3 

2 

Performance  Scales: 


MAINTAINING  PERSONAL  DISCIPLINE 


Occasional ly  shows  dis¬ 
respect  towards  superiors; 
often  fails  to  follow 
Army/unit  rules,  regula¬ 
tions  or  orders.  Creates 
disciplinary  problems. 

_ _ _  _ 

Rarely  exhibits  disre¬ 
spectful  behavior  towards 
superiors.  Almost  always 
follows  Army/unit  rules, 
regulations  or  orders. 

Always  treats  superiors 
with  respect.  Maintains 
high  level  of  personal 
integrity.  Obeys 
orders  quickly  and  with 
enthusiasm. 

1  2  3  4  5  6  7 

TASK  PROFICIENCY— MOS  SPECIFIC  TECHNICAL  SKILLS 


Does  not  display  the 
knowl edge/ski  1 1  required 
to  perform  many  core 
technical  skills. 


Displays  the  knowledge/ 
skill  required  to  perform 
most  core  technical  tasks 
properly,  but  may  need 
help  for  harder  tasks. 


Displays  the  knowledge/ 
skill  to  perform  all  core 
technical  tasks  properly. 


2  3 


4  5  6  7 


Sheet  No.  4 


Name 


MOS  116  OVERALL  PERFORMANCE  SCORE  SHEET 


Soldier 


No. 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 


Performance  Level 


Exercise  of  Leader* 
ship.  Effort,  end 
Self  Development 

6 

5 

2 

5 

2 

6 

4 
1 

5 
7 
3 

3 

4 
4 


Task  Proficiency— 
General 

Soldiering  Skills 

2 

5 

6 
3 

3 

5 
7 

4 

6 

4 

5 

3 

4 
1 
2 


Rank 

Order 


Performance  Scales: 


EXERCISE  OF  LEADERSHIP,  EFFORT  AND  SELF  DEVELOPMENT 


Fails  to  take  charge  when 
leadership  is  required  In 
unit.  Provides  little  or 
no  assistance  to  other 
unit  members.  Seldom 
exerts  effort  In  accom¬ 
plishing  many  job  assign¬ 
ments  and  tasks.  Gives 
up  easily  under  adverse 
conditions . 

Performs  satisfactorily  In 
leadership  situations 
where  what  Is  expected  is 
well  known.  When  asked, 
gives  help  anu  support  to 
fellow  soldiers.  Usually 
exerts  effort  to  perform 
most  Job  assignments  and 
tasks. 

Takes  charge  when  neces¬ 
sary  to  lead  unit;  leads 
the  squad  to  outstanding 
performance.  Does 
everything  possible  to 
assist  other  soldiers. 
Always  everts  consider¬ 
able  effort  in  performing 
all  job  assignments  and 
tasks. 

1  2  3  4  5  6  7 

TASK  PROFICIENCY— 6ENERAL  SOLDIERING  SKILLS 


Does  not  display  the 
knowledge/skill  required 
to  perform  many  general 
soldiering  tasks. 

i _ 

Displays  the  knowledge/ 
skill  required  to  perform 
most  general  soldiering 
tasks,  but  may  need  help 
for  harder  tasks. 

Displays  the  knowledge/ 
skill  to  perform  al 1 
general  soldiering 
skills. 

j  1  J  3  4  5  6  7 
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Name 


Sheet  No.  5 


MOS  1 IB  OVERALL  PERFORMANCE  SCORE  SHEET 


Soldier 

No. 


1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 


Performance  Level 


Maintaining 

Personal 

Discipline 


Military  Bearing/ 
Appearance  and 
Physical  Fitness 


6 

5 

2 

5 

2 

6 

4 

1 

5 
7 
3 

3 

4 
4 
3 


2 

5 

6 
3 

3 

5 
7 

4 

6 

4 

5 

3 

4 
1 
2 


Rank 


Performance  SCcleS: 


MAINTAINING  PERSONAL  DISCIPLINE 


Occasionally  shows  dis¬ 
respect  towards  superiors; 
often  fails  to  follow 
Army/unit  rules,  regula¬ 
tions  or  orders.  Creates 
disciplinary  problems. 

Rarely  exhibits  disre¬ 
spectful  behavior  towards 
superiors.  Almost  always 
follows  Army/unit  rules, 
regulations  or  orders. 

Always  treats  superiors 
with  respect.  Maintains 
high  level  of  personal 
integrity.  Opeys 
orders  quickly  and  with 
enthusiasm. 

1  2  3  4  5  6  7  i 

_ I 

MILITARY  BEAR  I N6/ APPEARANCE  AND  PHYSICAL  FITNESS 


Maintains  self  In  poor 
physical  condition.  Fails 
to  meet  military  standards 
for  dress  or  personal 
hygiene. 


Meets  Army  standards  of 
physical  fitness.  Dresses 
neatly  and  meets  Army 
standards  of  personal 
hygiene. 


Exceeds  Army  standards 
and  expectations  set  for 
physical  fitness.  Main¬ 
tains  excellent  personal 
hygiene  and  proper 
appearance. 


? 

Q. 

ft 

-I 


Hattie 


Sheet  No.  6 


NOS  1 IB  OVERALL  PERFORMANCE  SCORE  SHEET 


Soldier 

No. 


Performance  Level 


Task  Proficiency— 
NOS  Specific 
Technical  Skills 


Exercise  of  Leader¬ 
ship,  Effort,  and 
Self  Development 


Rank 

Order 


t'ertonnance  scales: 


TASK  PROFICIENCY— MOS  SPECIFIC  TICHNICAl  SKILLS 


Does  not  display  the 
knowl edge/ski  1 1  required 
to  perform  many  core 
technical  skills. 


Displays  the  knowledge/  Displays  the  knowledge/ 
skill  required  to  perform  skill  to  perform  all  core 
most  (.ore  technical  tasks  technical  tasks  properly, 
properly,  but  may  need 
help  f*»r  harder  tasks. 


EXERCISE  OF  LEADERSHIP,  EFFORT  AND  SELF  DEVELOPMENT 


Fails  to  take  charge  when 
leadership  is  required  in 
unit.  Provides  little  or 
no  assistance  to  other 
unit  members.  Seldom 
exerts  effort  in  accom¬ 
plishing  many  job  assign¬ 
ments  and  tasks.  Rives 
up  easily  under  adverse 
conditions . 


Performs  satisfactorily  in 
leadership  situations 
where  what  is  expected  is 
well  known.  When  asked, 
gives  help  and  support  to 
fellow  soldiers,  usually 
exerts  effort  to  perform 
most  job  assignments  and 
tasks. 


Takes  charge  when  neces-  ! 
sary  to  lead  unit;  leads 
the  squad  to  outstanding 
performance.  Dees 
everything  possible  to  | 
assist  other  soldiers. 
Always  exerts  consider¬ 
able  effort  in  performing 
all  job  assignments  and 
tasks. 


I 


Kane 


Sheet  No.  7 


MQS  11R  OVERALL  PERFORMANCE  SCORE  SHEET 


Soldier 

No. 


1 

2 

3 

M 

a 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 


Performance  Level 


Task  Proficiency—  Maintaining 

General  Personal 

Soldiering  Skills  Discipline 


6 

5 

2 

5 
2 

6 

4 
1 

5 
7 
3 

3 

4 
4 
3 


2 

5 

6 
3 

3 

5 

7 

4 

6 

4 

5 

3 

4 
1 
2 


Rank 

Order 


Performance  Scales: 


TASK  PROFICIENCY-GENERAL  SOLDIERING  SKILLS 


Does  not  di spl ay  the 
kcowl edge/ski i 1  required 
to  perform  many  general 
soldiering  tasks. 

Displays  the  knowledge/ 
skill  required  to  perform 
must  general  soldier! no 
tasks,  but  may  need  help 
for  harder  tasks. 

Displays  the  knowledge/ 
skill  to  perform  all 
general  soldiering 
skills. 

j  1  2  3  4  5  6  7  | 

1 _  ' 

MAINTAINING  PERSONAL  DISCIPLINE 


Occasionally  shows  dis¬ 
respect  towards  superiors; 
often  falls  tc  follow 
Army/unit  rules,  regula¬ 
tions  or  orders.  Creates 
disciplinary  problems. 


Rarely  exhibits  disre¬ 
spectful  behavior  towards 
superiors.  Almost  always 
fol 1 ows  Army/unl t  rul es , 
regu^tlons  or  orders. 


Always  treats  superiors 
with  respect.  Maintains 
high  level  of  personal 
integrity.  Obeys 
orders  qulcxly  and  with 
enthusiasm. 


Name 


Sheet  No.  ft 


MOS  118  OVERALL  PERFORMANCE  SCORE  SHEET 


Soldier 

No. 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 


Performance  Level 


Military  Bearing/ 
Appearance  and 
Physical  Fitness 

6 

5 

2 

5 
2 

6 

4 
1 

5 
7 
3 

3 

4 
4 
3 


Task  Proficiency— 
MOS  Specific 
Technical  Skills 

2 

5 

f 

3 

3 

5 
7 

4 

6 

4 

5 

3 

4 
1 
2 


Rank 

Order 


Performance  Scales: 


MILITARY  BEAR I N€/ APPEARANCE  AND  PHYSICAL  FITNESS 


Maintains  self  In  poor 
physical  condition.  Falls 
to  meet  military  standards 
for  dress  or  personal 
hygiene. 



Meets  Army  standaros  of 
physical  fitness.  Dresses 
neatly  and  meets  Arny 
standards  of  personal 
hygiene. 

Exceeds  Army  standards 
and  expectations  set  for 
physical  fitness.  Main¬ 
tains  excellent  personal 
hygiene  and  proper 
appearance. 

1  2  3  4  5  6  7 

i -  ..  — - - —  -  -  -  -  ■  1 

TASK  PROFICIENCY— MOS  SPECIFIC  TECHNICAL  SKILLS 


Does  not  display  the 
knowledge/skill  required 
to  perform  many  core 
techrical  skills. 

Displays  the  knowledge/ 
skill  required  to  perform 
most  core  technical  tasks 
properly,  but  may  need 
help  for  harder  tasks. 

Displays  the  knowledge/ 
skill  to  perform  all  core 
technical  tasks  properly. 

1  2  3  4  5  6  7 

0-16 


Sheet  No.  9 


Wane 


«0S  116  OVERALL  PERFORMANCE  SCORE  SHEET 


Performance  Level 

Exercise  of  Leader¬ 

Maintaining 

Soldier 

ship,  Effort,  end 

Personal 

Rank 

Mo. 

Self  Development 

Discipline 

Order 
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6 

2 

2 

5 

5 

3 

2 

6 

4 

5 

3 

s 

2 

3 

6 

6 

5 

7 

4 

7 

8 

1 

4 

9 

5 

6 

10 

7 

4 

11 

3 

5 

12 

3 

3 

13 

4 

4 

14 

4 

1 

15 

3 

2 

. 

Performance  Scales: 

EXERCISE  OF  LEADERSHIP,  EFFORT  AMD  SELF  DEVELOPMENT 


Fails  to  take  charge  when 
leadership  Is  required  In 
unit.  Provides  1 Ittle  or 
no  assistance  to  other 
unit  members.  Seldom 
exerts  effort  in  accom¬ 
plishing  many  job  assign¬ 
ments  and  tasks.  Gives 
up  easily  under  adverse 
conditions. 

Performs  satisfactorily  in 
leadership  situations 
where  what  Is  expected  Is 
well  known.  When  asked, 
gives  help  and  support  to 
fellow  soldiers.  Usually 
exerts  effort  to  perform 
most  job  assignments  and 
tasks. 

Takes  charge  when  neces¬ 
sary  to  lead  unit;  leads 
the  squad  to  outstanding 
performance.  Does 
everything  possible  to 
assist  other  soUiers. 
Always  exerts  consider¬ 
able  effort  In  performing 
all  job  assignments  and 
tasks. 

1  2 

3  4  5 

6  7 

MAINTAINING  PERSONAL  DISCIPLINE 

Occasionally  shows  dis¬ 
respect  towards  superiors; 
often  fails  to  follow 
Army/unit  rules,  regula¬ 
tions  or  orders.  Creates 
disci inary  pr  ^blems. 

Rarely  exhibits  disre¬ 
spectful  behavior  towards 
superiors.  Almost  always 
follows  Army/unit  rules, 
regulations  or  orders. 

Always  treats  superiors 
with  respect.  Maintains 
high  level  of  personal 
Integrity.  Obeys 
orders  quickly  and  with 
enthusiasm. 

1  2 

3  4  5 

6  7 

D~  1 7 


Name 


Sheet  No.  10 


WS  113  OVERALL  PERFORMANCE  SCORE  SHEET 


Soldier 

No. 


1 

2 

3 

4 
b 

e 

7 

8 
9 

10 

11 

12 

13 

14 

15 


Performance  Level 


Task  Proficiency— 
General 

Soldiering  Skills 

€ 

5 

2 

5 
2 

6 

4 
1 

5 
7 

3 

4 

4 

3 


Military  Bearing/ 
Appearance  and 
Physical  Fitness 

2 

5 

6 
3 

3 

5 
7 

4 

6 

4 

5 

3 

4 
1 
2 


Rank 

Order 


Performance  Scales: 
•  >■ '  '  ■  ■■  — 


TASK  PROFICIENCY— GENERAL  SOLDIERING  SKILLS 


Does  riot  display  the 
knowledge/skill  required 
to  perform  many  general 
Soldier ing  tasks. 

Displays  the  knowledge/ 
skill  required  to  perform 
mosf  general  soldiering 
tasks,  but  may  need  help 
for  harder  tasks. 

Displays  the  knowledge/ 
Skill  to  perform  all 
general  soldiering 
skills. 

|  1  2  3  4  5  C  7  ! 

MILITARY  BEARING/ APPEARANCE  AND  PHYSICAL  FITNESS 


Maintains  self  In  poor 
physical  condition.  Fails 
to  meet  military  standards 
for  dress  or  personal 
hygiene. 


Meets  Army  standards  of 
physical  fitness.  Dresses 
neatly  and  meets  Army 
standards  of  personal 
hygiene. 


Exceeds  Army  standards 
and  expectations  set  for 
physical  fitness.  Main¬ 
tains  excellent  personal 
hygiene  and  proper 
appearance. 


Appendix  E 

UTILITY  WORKSHOP  INSTRUCTIONS  AND  MATERIALS 


E-l 


Assigning  Utility  Values  to  Any  Beenrtts 


Overview 

Many  people  have  suggested  that*  all  other  things  being  equal,  newly 
accessioned  recruits  differ  In  term  of  their  predicted  overall  value  of 
utility  to  the  Artsy  for  accomplishment  of  particular  A ray  Missions.  This  is 
not  an  argument  that  certain  recruits  are  not  needed.  Rather,  It  is  an 
assertion  that,  given  some  base  line,  adding  certain  recruits  to  some  MOS  has 
relatively  greater  utility  to  the  Army  than  adding  ether  recruits  to  other 
MOS. 


In  order  to  allow  a  computerized  enlisted  personnel  selection  and 
classification  system  to  operate  In  the  best  Interest  of  the  Army,  the 
decisions  aade  by  the  system  must  reflect  the  best  judgment  of  experienced 
Army  officers.  ‘  To  Inform  the  computerized  processes  Involved  In  selecting 
and  classifying  applicants  for  enlistment,  you  will  be  asked  to  judge  the 
relative  priority  that  the  system  should  place  on  filling  different  MOS  with 
recruits  having  different  predicted  performance  levels.  This  Is  not  to  Imply 
that  some  MOS  should  have  no  newly  assigned  recruits  or  only  low  or  high 
level  recruits,  but  that  the  system  should  attempt  to  meet  the  most  critical 
Army  personnel  needs  first. 


Purpose  of  Workshop 

In  order  to  determine  how  best  to  measure  MOS  personnel  classification 
priorities,  we  are  trying  out  various  methods  of  obtaining  judgments  of 
experienced  Army  officers.  In  this  workshop  we  will  try  out  three  methods. 
The  methods  call  for  Increasingly  complex  judgments  concerning  the  value  or 
usefulness  of  classifying  recruits  Into  different  MOS.  In  the  first 
procedure,  you  will  be  asked  to  sort  recruits  Into  utility  categories  based 
on  relative  classification  priorities.  In  the  second  procedure,  you  will 
judge  the  value  of  recruits  relative  to  the  value  of  a  specified  Infantryman 
(118)  recruit.  The  third  procedure  Involves  classifying  groups  of  recruits 
of  various  predicted  performance  levels  into  a  limited  number  of  MOS.  In 
making  all  the  judgments  called  for,  please  consider  tiie  likely  usefulness  of 
the  recruits  at  the  given  performance  level  in  that  MOS  in  helping  achieve 
the  Army's  mission  in  comparison  to  other  recruits  at  other  performance 
levels  In  other  MOS. 


Assumptions 


To  help  assure  that  all  workshop  participants  are  starting  from  the  same 
place,  we  would  like  you  to  make  the  following  assumptions  when  making  your 
judgments: 

(1)  The  military  context  for  which  the  utility  of  the  recruits 
Is  being  considered  Is  as  follows: 


The  world  Is  In  a  period  of  heightened  tensions.  Thera  Is 
ar.  Increasing  probability  that  hostilities  will  break  out  In 
Europe,  Asia,  the  Caribbean,  Latin  America  and  Africa.  The 
Army's  mission  is  to  support  U.S.  treaty  obligations  and  to 
help  defend  the  borders  of  allied  and  friendly  nations. 
Sow  of  the  potential  enesiies  have  i.  leer  and  chemical 
capability.  Air  parity  does  exist  between  allied  forces  and 
potential  hostile  nations.  U.S.  Army  training  and  other 
preparatory  activities  have  been  substantially  Increased. 
Host  combat  and  associated  support  units  are  participating 
In  frequent  field  exercises.  Most  units  are  being  ectlvely 
resupplied. 

(2)  The  field  strength  of  all  MQS  overseas  Is  the  sane — 70 
percent. 

(3)  Troop  replacement  needs  resulting  from  any  anticipated 
wartime  casualties  will  be  handled  separately  by  the 
computerized  personnel  selection  and  classification  system. 
(That  Is,  the  relative  priorities  should  reflect  only  the 
likely  usefulness  of  recruits  at  given  predicted  performance 
levels  in  helping  to  achieve  the  Amy's  mission.) 

(4)  The  measures  used  to  predict  performance  Include  not  only 
aptitude  scores  (taken  from  the  Armed  Services  Vocational 
Aptitude  Battery)  but  also  tests  of  psychomotor  skills,  work 
history.  Interests,  motivation,  and  other  Indexes  that 
prcu'Cb  overall  M35  parfcrsancs. 

(5)  The  overall  MOS  performance  measure  for  each  MOS  represents 
an  optimally  weighted  (for  that  MOS)  combination  of  several 
performance  factors.  Thus,  recruits  at  the  highest 
predicted  performance  level  (90th  percentile)  In  each  MOS 
are  more  likely  to  be  dependable,  proficient  in  MOS  tasks, 
know  the  facts  and  procedures  required  to  dt  their  jobs, 
perform  more  effectively  under  adverse  or  difficult 
conditions,  avoid  disciplinary  problems,  provide  support  to 
fellow  soldiers,  and  ue  more  physically  fit. 

(6)  Th*  predicted  performance  levels  for  the  recruits  are 
accurate.  That  Is,  tht  recruits  will  actually  perform  at 
the  predicted  levels, 

(7)  The  spread  or  amount  of  variation  In  predicted  performance 
Is  equal  in  each  MOS. 


E-3 


DIRECTIONS  FOR  PROCEDURE  A:  SORTING  ARMY  RECRUITS 
ON  PRIORITY  FOR  FILLING  STRENGTH  REQUIREMENTS 


In  the  first  procedure  to  be  tried  out  today  you  will  be  given  «  set  of 

cerds  On  each  card  there  Is  t  short  description  of  en  MOS  (first  tour  or 

Ski  11  Level  10)  taken  from  AR  611-201.  which  gives  descriptions  of  all  Army 
KOS.  Also  on  each  card  Is  the  predicted  performance  level  In  that  MOS  of  the 
given  recruit.  The  levels  have  been  set  at  the  10,  30,  50,  70  or  90th 
percentile  level  and  are  based  on  the  lelection/classlficatlon  measures 

available  on  the  recruits.  All  percentiles  are  the  predicted  percentiles  of 
performance  of  the  recruits.  If  all  recruits  were  rank  ordered  in  terms  of 
their  predieted  performance  In  'tfie  given  MOS  without  regard  to  current 
cut-off  scores.  Note  that  a  10th  percentile  performance  level  signifies  .low 
performance  and  a  90th  percentile  performance  level  signifies  hi  on 
performance.  Also  please  note  that  the  percentiles  refer  to  percenti tes 

within  all  newly  accessioned  recruits  assuming  that  the  recruits  were  rank 
ordered  In  terms  of  their  predicted  performance  scores  for  the  given  MOS. 

The  recruits  on  the  cards  have  been  tentatively  assigned  to  56  MOS 
without  regard  to  current  cut-off  scores.  The  judgment  task  Is  to  sort  280 
cards  (S6  MOS  X  5  predicted  performance  levels)  Into  7  piles  or  categories 
reflecting  the  relative  utility  of  the  recrufts  described  on  the  cards.  The 
categories  are: 

High  positive  utility  would  probably  result  If  these  recruits  were 
placed  in  these  MOS. 

Between  moderate  and  high  utility  would  probably  result  If  these 
recruits  were  placed  In  these  MOS. 

Moderate  utility  would  probably  result  If  these  recruits  were  placed  In 
these  MOS. 

Between  low  end  moderate  utility  would  probably  resurlt  If  these  recruits 
were  placed  In  these  MOS. 

Low  positive  utility  would  probably  result  If  the?e  recruits  were  placed 
in  these  MOS, 

Advantages  of  placing  these  recruits  In  these  MOS  would  probably  be 
equal  to  the  disadvantages  (expected  utility  •  0). 

Negative  utility  would  probably  result  1?  these  recruits  were  placed  In 
these  R3S.  (Any  positive  contribution  would  probably  be  outweighed  by 
problems  associated  with  low  levels  of  overall  performance) . 


Specific  Directions: 

(1)  Familiarize  yourself  with  the  MOS  by  examining  the  descriptions  on 
the  cards  and  with  the  above  descriptions  of  the  7  piles.  You  will 
be  provided  with  e  set  of  label  cards  containing  the  pile  descrip¬ 
tions. 
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(2)  Then  sort  the  cards  (which  tr«  In  renioa  order)  Into  the  7  piles. 
You  ere  free  to  piece  ss  eany  ctr**\  it  you  like  In  any  one  pile. 
If  you  do  not  feel  thet  you  ere  fa^llar  enough  with  eny  given  MOS 
to  aeke  e  coteperetlve  evaluation,  p’^oe  the  cards  for  thet  MOS  In 
an  eighth,  unrated  pile. 

(3)  Uhen  you  hove  finished  your  first  sor  \ ,  go  through  the  piles  care¬ 
fully,  asking  any  changes  In  the  sorting  you  feel  are  appropriate. 

(4)  Uhen  you  art  satisfied  with  your  sorts,  pleaiC  place  the  appro¬ 
priate  label  card  on  top  of  each  pile  ani  secure  each  pile  with  a 
rubber  band.  Then  use  a  rubber  band  tc  bind  the  piles  together 
with  your  Mac*  Card  on  top. 


Thank  you  for  your  cooperation. 
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DIRECTIONS  FDR  PROCEDURE  B:  SETTIM  THE 
UTILITY  OF  ARMY  RECRUITS  RELATIVE  TO  THE  VALUE 
OF  A  ROTH  PERCENTILE  IBFAMTRYWJJ  -111 


In  the  second  procedure  you  will  try  out  today,  you  will  be  Riven  a  set 
of  60  card*.  These  cards  are  a  subset  of  the  cards  you  sorted  earlier.  The 
cards  cover  12  HOS,  with  recruits  at  the  S  predicted  performance  levels  you 
evaluated  earlier.  The  judgment  task  is  to  assign  a  numerical  utility  value 
to  each  of  the  60  recruits.  The  values  that  you  assign  will  be  proportionate 
to  the  value  assigned  to  a  90th  percentile  Infantryman  recruit.  In  other 
words,  the  overall  worth  or  utility  of  tbo  90th  percentilo  Infantryman 
roeruit  will  bo  used  es  a  yardstick  and  the  worth  of  ell  other  recruits  will 
be  Judged  in  relationship  to  this  Infantryman  recruit.  (This  is  comparable 
to  the  use  of  a  given  platinum  bar  as  the  defined  length  of  1  motor  or  100 
centimeters  In  the  metric  system.)  In  this  case  tha  value  of  the  90th 
percentile  Infantryman  will  be  set  at  100. 

In  making  your  judgments,  please  make  tha  same  set  of  assumptions  as 
were  nade  in  the  previous  procedure.  Please  review  these  assumptions  before 
beginning  the  judgment  task. 


Specific  Directions; 

(1)  Faalllarlxe  yourself  with  the  12  KQS  by  examining  the  descriptions 
provided. 

(2)  Write  the  value.  100,  on  the  DOth  percentile  Infantryman  card  which 
is  on  top  of  the  deck  you  Just  received.  (The  ocher  »ru*  In  ih« 
deck  are  In  random  order.) 

(3)  Then  take  each  of  the  other  cards  and  assign  the  soldier  a  utility 
value  which  reflects  the  worth  of  each  recruit  relative  to  the 
value  <?f  the  90th  percentile  Infantryman  recruit  which  has  a  worth 
of  100.  You  may  assign  higher  values  than  100  or  even  negative 
values  to  one  or  more  recruits  if  you  wish  (see  scale  provided  on 
next  page).  In  other  words,  you  are  free  to  assign  any  number  that 
reflects  the  relative  worth  of  the  recruit  being  evaluated.  Write 
the  values  you  assign  directly  on  the  cards  In  the  lower  right  hand 
corner. 

(♦)  When  you  have  gone  through  the  deck  once,  please  arrange  the  cards 
In  numerical  order  from  lowest  to  highest  value. 

(5)  Then  go  through  the  cards  once  more  and  change  any  assigned  value 
that  you  feel  is  out  of  line  with  the  others  (with  the  exception  of 
the  value  of  100  assigned  to  the  90th  percentile  Infantrymen). 
Please  refer  to  the  attached  Utility  Rating  Scale  to  help  resolve 
any  scaling  problems. 


UTILITY  RATING  SCALE* 


JL50  -  Recruit  1*  worth  SOX  mom  to  tht  Amy  than  an  Infantryman  (118)  meruit 
At  the  9Cth  perctntile  level  of  predicted  performance. 

125  -  Recruit  It  worth  25*  sort  to  the  Army  then  in  Infantryman  (Hi)  meruit 
At  th«  90th  percentile  of  predicted  performance. 

100  -  Utility  to  the  Amy  of  Ah  Infantryman  (1IB)  meruit  At  the  90th 
percentile  level  of  predicted  performance  in  the  scenario  described. 

75  -  Utility  of  this  meruit  it  3/4  thAt  of  in  Infantrymen  (118)  meruit  At 
the  90th  percentile  level  of  predicted  perf omened. 

50  -  Utility  of  thi*  meruit  i*  1/2  thet  of  ah  Infantryman  meruit  At  the 
SOth  percentile  level  of  pmdicted  perfomence. 

25  **  Utility  of  this  recruit  1*  1/4  thAt  of  An  Infantryman  meruit  At  the 
90th  percentile  level  of  predicted  perfomence. 

0  •  Advantages  of  having  this  meruit  In  the  scenario  described  am  equal 
to  the  disadvantages. 

-25  *  Use  of  this  meruit  would  result  In  a  net  loss  to  the  Amy  equal  to  the 
gain  that  would  result  from  using  a  meruit  with  a  utility  value  of  25. 

-50  -  Use  of  this  meruit  would  result  In  a  net  loss  to  the  Amy  equal  to  the 
gain  that  would  msult  from  using  a  recruit  with  a  utility  value  of  50. 


*  Please  note  that  values  higher  than  ISO  and  lower  than  -50  can  he 
assigned.  Also,  any  value  in  between  the  scale  points  given  can  be 
assigned,  that  is,  you  am  jiot  restricted  to  the  values  appearing  on  the 
above  scale. 
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Appendix  F 


AVERAGE  RATIO  SCALE  UTILITIES  FOR  273  AOS  BY  PERFORMANCE  PERCENTILE 
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Average  Ratio  Scale  Utilities  by  Perforaanca  Percentile 
_ _ Percentile _ 


m 

1 ft 

1ft 

5ft 

2ft 

1ft 

008 

-13.2 

22.3 

52.2 

74.9 

88.2 

OIK 

-19.3 

22.7 

33.4 

77.3 

94.3 

03C 

12.7 

35.0 

57.1 

63.0 

67.1 

050 

-9.8 

29.4 

53.7 

79.2 

113.5 

05N 

-19.0 

27.3 

52.5 

84.7 

.  112.6 

05X 

-3.2 

19.8 

53.1 

88.1 

110.7 

118  * 

4.5 

38.7 

61.7 

82.3 

100.0 

11C 

5.0 

35.0 

70.6 

97.4 

111.8 

11H 

13.6 

49.0 

65.2 

93.3 

117.9 

m 

8.4 

41.5 

74.8 

90.8 

103.0 

126 

5.1 

47.2 

63.3 

86.8 

112.6 

12C 

8.5 

42.9 

70.8 

78.0 

38.1 

12£ 

-17.5 

20.8  . 

41.3 

75.7 

95.9 

12F 

6.5 

33.1 

66.2 

85.5 

101.9 

138 

20.8 

35.0 

67.1 

85.2 

111.7 

13C 

-17.  b 

32.8 

53.2 

80.3 

«  «  r  a 
IID.C 

13E  * 

-3.3 

35.7 

64.6 

90.5 

111.9 

13F 

15.5 

35.0 

59.1 

85.2 

105.0 

13M 

16.9 

33.4 

59.3 

89.1 

97.0 

13R 

-11.2 

21.7 

55.2 

83.0 

116.9 

13T 

-13.2 

18.7 

56.2 

66.4 

79.3 

150 

-7.5 

35.5 

55.8 

81.5 

105.6 

15E 

0.6 

23.5 

52.5 

80.6 

117.9 

15J 

-21.8 

8.5 

57.3 

78.0 

120.1 

%  £.  fi 

iUU 

n  c 

28.2 

Cl  t 

AC.? 

101.9 

16E 

-3.1 

26^4 

50.5 

81.9 

91.7 

16F 

-0.1 

29.5 

58.2 

92.7 

106.9 

156 

-7.5 

27.3 

57.8 

68.3 

93.1 

16H  * 

-8.4 

31.0 

57.3 

86.3 

107.8 

16J 

-6.3 

25.8 

51.3 

73.5 

115.2 

16P 

-7.2 

31.2 

57.3 

80.5 

107.7 

16R 

15.5 

32.8 

49.3 

80.3 

93.1 

16S 

-8.2 

33.4 

40.3 

77.3 

105.6 

16T 

-11.6 

22.3 

60.2 

92.7 

116.8 

*  One  of  12  HOS  assigned  actual  ratio  scales 
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m 

1ft 

2ft 

2ft 

2ft 

*ft 

178 

-0.6 

33.4 

52.2 

89.1 

105.6 

17C 

-0.6 

31.7 

48.8 

91.7 

108.7 

190 

1.6 

35.0 

82.3 

92.7 

111.8 

19E 

8.4 

45.6 

68.3 

98.0 

108.2 

19K 

5.1 

47.2 

72.7 

95.6 

110.0 

216 

-21.8 

8.5 

51.0 

80.5 

110.7 

21L 

-17.5 

28.2 

51.3 

75.7 

111.7 

22L 

-8.2 

24.5 

52.2 

84.2 

84.2 

22H 

-13.2 

20.5 

50.2 

83.7 

106.9 

23 H 

-22.2 

25.3 

45.6 

66.2 

103.0 

23U 

-16.3 

17.6 

59.7 

70.8 

107.5 

24C 

-13.3 

16.0 

59.5 

93.5 

113.9 

24E 

-13.6 

30.5 

45.3 

85.2 

111.7 

246 

-13.6 

26.4 

45.4 

81.9 

99.8 

24H 

-11.6 

25.8 

54.2 

85.9 

111.8 

24J 

-14.7 

23.2 

49.6 

68.3 

103.0 

24K 

-19.0 

17.6 

56.1 

80.6 

110.0 

24L 

-19.6 

12.3 

42.9 

78.0 

116.9 

24H 

-17.5 

25.8 

53.8 

80.3 

108.3 

24N 

-5.6 

24.5 

52.2 

81.9 

1Z1.8 

24P 

-6.3 

20.8 

47.3 

78.0 

115.2 

24Q 

-16.4 

22.3 

52.2 

77.1 

106.9 

24S 

-9.8 

21.1 

45.6 

62.0 

88.4 

24T 

-19.0 

17.6 

50.7 

78.6 

112.6 

24U 

-17.5 

23.6 

59.5 

75.5 

107.7 

25L 

-10.0 

24.1 

56.2 

85.9 

109.3 

26C 

4.1 

33.4 

47.1 

84.2 

99.8 

26E 

-14.8 

20.5 

56.2 

77.1 

95.0 

Z6F 

-12.2 

a  ^  » 

e 

/  #  ®  V 

oo  n 

»  W 

26H 

-13.8 

23.5 

49.0 

80.6 

112.6 

26K 

-13.3 

23.6 

44.9 

80.5 

113.8 

26Q 

-9.8 

25.3 

45.6 

77.0 

108.2 

26T 

-9.9 

15.5 

51.3 

67.1 

80.3 

26V 

-13.6 

20.8 

48.8 

77.3 

111.8 

26Y 

-13.2 

20.5 

48.3 

74.9 

109.3 
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pgrccnt.lte 


SSL 

SSL 

la 

SSL 

27B 

-8.2 

26,4 

50.5 

77.3 

102.7 

27C 

-13.2 

20.5 

58.2 

83.7 

104.5 

270 

-17.1 

21.1 

43.5 

62.0 

93.1 

27E 

-13.8 

31.0 

50.7 

91.1 

115.2 

27F 

-21.8 

17.9 

46.9 

80.5 

110.7 

276 

-17.5 

30.  b 

45.3 

60.3 

101.9 

27L 

-10.8 

31.7 

45.4 

81.9 

115.0 

27M 

-10.0 

15.2 

56.2 

85.9 

111.8 

27N 

-14.7 

27.3 

47.6 

68.3 

98.0 

27P 

-14.7 

16.9 

43.5 

66.2 

83.8 

m 

-11.3 

27.3 

52.5 

86.8 

105.0 

29F 

-21.8 

14.1 

55.2 

90.8 

104.8 

290 

-9.2 

25.5 

53.1 

83.0 

90.8 

29M 

-9.9 

23.3 

53.2 

78.0 

108.3 

29H 

-6.3 

28.2 

47.3 

75.7 

101.9 

29S 

-5.6 

24.5 

54.0 

89.1 

97.0 

31C 

-6.4 

25.4 

54.3 

86.8 

97.9 

31K 

-3.2 

27.4 

57.3 

85.5 

96.2 

31K 

-6.3 

32,8 

51.3 

82.7 

53  1 

31N 

-14.8 

18.7 

5?. 2 

77.1 

102.1 

31V 

-5.1 

or 

cc  o 

S3. 4 

105  6 

32D 

-6.4 

19.6 

54.3 

76.6 

93.3 

33P 

-11.6 

18.7 

50.2 

83.7 

116.8 

33Q 

-17.1 

10.5 

45.6 

79.2 

108.2 

33R 

-24.4 

21.6 

52.5 

84.7 

105.0 

33T 

-11.2 

21.7 

48.9 

80.5 

110.7 

34L 

-15.4 

19.8 

53.1 

78.0 

116.9 

34T 

-13.6 

28.2 

49.3 

80.3 

108.3 

34  y 

-21.5 

23.3 

45.3 

80.3 

105. C 

35F 

-5.6 

33.4 

47.1 

86.6 

i05.6 

35G 

-5.6 

20.8 

47.1 

75.1 

99.8 

35n 

-6.7 

20.5 

48.3 

70.6 

85.9 

35K 

-19.6 

19.0 

47.6 

79.2 

98.0 

35L 

-21.6 

17.6 

56.1 

84.7 

107.5 

35M 

-21.8 

23.6 

46.9 

70.8 

110.7 

35R 

-21.5 

20.8 

45.3 

75.7 

111.7 

36C 

4.6 

40.9 

48.9 

75.5 

83.0 

361 

-3.2 

28.2 

61.2 

66.9 

99.8 

36M 

6.4 

31.7 

55.7 

75.1 

97.0 

39B 

-11  6 

24.1 

48.3 

72.8 

99.7 
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m 

it 

& 

& 

ISt 

J £ 

418 

-14.fi 

22.3 

44.4 

72.8 

82.5 

41C 

-19.3 

28.2 

52.2 

81.9 

105.6 

41E 

6.2 

31.4 

47.6 

68.3 

72.6 

41J 

0.6 

27.3 

50.7 

68.9 

76.6 

42C 

-5.6 

26.4 

47.1 

63.1 

86.6 

420 

-13.2 

22.3 

48.3 

74.9 

83.7 

42E 

-2.8 

25.3 

45.6 

66.3 

83.6 

W3E 

-19.6 

14.8 

43.5 

72.6 

110.8 

43M 

6.2 

27.3 

64.1 

62.0 

72.6 

44B 

C.S 

35.0 

57.3 

70.8 

90.8 

44E 

-9.9 

35.0 

51.3 

80.3 

96.9 

458 

6.8 

35. 2 

51.3 

85.2 

95.9 

45D 

6.4 

33.4 

55.7 

75.1 

97.0 

45E 

-13.2 

29.5 

60.2 

90.4 

114.3 

5V, 

-11.6 

24.1 

58.2 

90.4 

116.8 

45K 

-7.5 

27.3 

59.9 

79.2 

108.2 

45L 

-1.7 

31.0 

59.7 

86.8 

115.2 

45N 

-7.2 

31.2 

63.9 

83.0 

116.9 

45T 

-13.6 

25.8 

47.3 

85.2 

115.2 

46N  * 

-lb.O 

25.9 

57.5 

93.1 

119.5 

51B 

22.7 

36.9 

54.0 

66.9 

59.3 

51C 

-8.3 

20.5 

46.4 

72.8 

83.7 

51G 

3.3 

31.3 

52.2 

66.4 

74.9 

51K 

10.5 

25.3 

59.  S 

66.2 

70.5 

SiM 

10.5 

33.4 

53.7 

74.8 

79.2 

SIN  # 

-7.6 

29.3 

51.5 

69.6 

81.5 

SIR 

-1.7 

19.6 

47.2 

67.0 

84.7 

52C 

1  o 

1  .o 

*41  ^ 

Ji  •  / 

CO  7 

66.9 

Ql  .7 

520 

-3.4 

24.1 

60.2 

81 1 5 

92.7 

52F 

-14.7 

27.3 

47.8 

66.2 

95.6 

52G 

-11.2 

25.5 

53.1 

73.1 

85.5 

54C 

9.8 

35.0 

55.1 

80.3 

87.7 

54E 

1.8 

31.7 

45.4 

89.1 

108.7 

SSB 

0.6 

40.1 

61.5 

82.6 

100.2 

550 

-13.2 

20.5 

54.2 

70.6 

97.4 

55G  * 

-24.0 

15.7 

56.4 

94.1 

124.8 

55R 

-1.2 

33.1 

59.5 

73.1 

90.8 

°  f;ne  of  12  NOS  assigned  tctual  ratio  scales 
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1 


m 

1ft 

1ft 

ftft 

2ft 

Jft 

57E 

23.5 

25.4 

52.5 

68.9 

72.7 

57F 

2.7 

35.0 

63.9 

75.5 

75.5 

57H 

12.7 

41.3 

53.2 

85.2 

90.4 

61B 

8.6 

38.6 

42.0 

75.1 

89.1 

61C 

•1.7 

17.6 

49.0 

65.2 

88.9 

62S 

0.8 

31.2 

66.2 

85.5 

96.2 

62E 

-6.7 

31.3 

60.2 

85.9 

92.7 

62F 

-0.5 

37.5 

57.8 

83.8 

86.1 

620 

0.6 

27.3 

45.4 

63.3 

80.6 

62K 

0.8 

42.9 

61.7 

68. 5 

83.0 

62.) 

0.5 

28.2 

57.1 

71.3 

82.7 

63B 

0.5 

35.0 

55.1 

87.7 

111.7 

63D 

-0.6 

38.6 

61.2 

75.1 

97.0 

63E 

-8.3 

31.3 

64.4 

92.7 

114.3 

636 

-C.5 

29.4 

53.7 

81.5 

96.0 

63H 

13.6 

43.7 

65.2 

88.9 

110.0 

63J 

2.7 

37.0 

61.7 

83.0 

56.2 

63N 

-21.5 

35.0 

57.1 

78.0 

1)5.2 

63S 

-3.1 

28.2 

63.1 

84.2 

86.6 

OT 

UJ  1 

-6  7 

31.3 

62.3 

90.4 

114.3 

63W  * 

-0.6 

32.3 

60.5 

85.5 

102.  / 

63Y 

6.2 

35.5 

62.0 

88.4 

105.6 

64C 

22.7 

48.8 

52.2 

81.9 

79.6 

65B 

2.9 

27.3 

49.0 

61.5 

72.7 

65D 

16.0 

29.3 

66.2 

70.8 

65.5 

65E 

-13.6 

23.3 

41.3 

69.2 

87.7 

65F 

-22.3 

24.5 

43.7 

70.9 

77.3 

65G 

-9.9 

32.8 

43.3 

73.5 

85.2 

65H 

-3.4 

29.5 

58.2 

77.1 

54.2 

65J 

12.7 

29.4 

^ 

01 .  / 

r  r\ 

UO.  J 

Ol  c 

V*  «  w 

65K 

-8.8 

25.4 

47.2 

67.0 

93.3 

676 

-12.2 

19.0 

53.7 

70.5 

103.0 

67H 

-24.4 

21.6 

49.0 

80.6 

110.0 

67N 

-24.0 

8.5 

46.9 

80.5 

107.7 

67R 

-25.7 

20.8 

47.3 

82.7 

115.2 

67S 

-22.3 

20.8 

52.2 

91.7 

108.7 

67T 

-1\B 

IB.  7 

48.3 

G3.7 

114.3 

67U  * 

-17.9 

23.0 

56.6 

936 

119.7 

67V 

-14.7 

21.1 

51.7 

70.5 

105.6 

67X 

-16.3 

25.4 

53.7 

GO. 6 

120.6 

67Y 

-24.0 

21.7 

48.9 

85.5 

113.8 

*  One  of  12  MOS  assigned  actual  ratio  scales 
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m 

& 

m 

686 

-25.7 

9.8 

41.3 

80.3 

115.2 

680 

-25.4 

227 

42.0 

77.3 

108.7 

68F 

-24.7 

23.2 

45.6 

77.0 

108.2 

686 

-16.4 

20.5 

46.4 

81.5 

102.1 

63H 

-14.7 

19.0 

49.6 

78.2 

105.6 

66J 

-16.3 

21.6 

57.9 

82.6 

115.2 

68M 

-13.3 

21.7 

46.9 

83.0 

113.8 

71C 

-0.6 

22.7 

47.1 

56.9 

97.0 

710 

-10.0 

29.5 

48.3 

70.6 

88.2 

716 

-2.8 

33.4 

53.7 

74.8 

90.8 

71L 

0.6 

27.3 

54.3 

70.8 

86.8 

71M 

8.5 

33.1 

61.7 

66.2 

S3.0 

71N 

-10.8 

28.2 

50.5 

86.6 

115.0 

71Q  * 

-9.9 

16.9 

36.7 

51.4 

62.6 

71R 

-4.0 

27.3 

43.7 

59.7 

67.0 

72E 

-6.3 

30.5 

63.0 

80.3 

101.9 

726  * 

-5.6 

31.1 

55.1 

82.6 

98.8 

72rl 

-19.3 

24.5 

35.2 

73.0 

105.6 

73C 

-6.3 

37.1 

49.3 

78.0 

101.9 

i  JU 

oo  o 
•  «/ 

on  a 

kW  •  W 

•»n  * 

fcfl  O 

89  =  1 

74D 

-5.6 

18.9 

47.1 

79.6 

105.6 

74F 

-16.4 

18.7 

44.4 

70.6 

104.5 

75B 

-0.1 

24.1 

56.2 

72.8 

85.9 

75C 

1.7 

25.3 

55.8 

79.2 

103.0 

750 

0.6 

23.5 

59.7 

67.0 

86.8 

75E 

4.6 

25.5 

61.7 

85.5 

93.5 

75F 

-2.9 

30.5 

47.3 

80.3 

105.0 

76C 

-3.4 

33.1 

56.2 

81.5 

77.1 

76J 

-7.5 

23.2 

55.0 

66.3 

93.1 

76P 

-4.0 

27.3 

54.3 

76.6 

88.9 

76V 

-3.2 

40.9 

55.2 

78.0 

90.8 

76W 

15.5 

39.2 

55.1 

73.5 

82.7 

76X 

6.4 

28.2 

52.2 

79.6 

91.7 

76Y 

-5.0 

33.1 

66.4 

81.5 

92.7 

81B 

2.7 

27.4 

51.0 

68.5 

93.5 

81C 

0.5 

23.3 

55.1 

78.0 

98.9 

81E 

-6.7 

22.3 

60.2 

66.4 

62.3 

81Q 

1.7 

35.5 

59.9 

81.5 

110.8 

*  One  of  12  MOS  assigned  actual  ratio  scales 
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.11 


mi 

1ft 

1ft 

K 

Zft 

2ft 

82B 

-1.7 

31.0 

49.0 

72.7 

86.8 

82C 

-5.1 

37.5 

51.7 

86.1 

108.2 

820 

-11.2 

19.8 

53.1 

78.0 

96.2 

83E 

-0.6 

26.4 

55.7 

68.9 

77.3 

83F  * 

-6.0 

23.4 

37.8 

54.7 

57.7 

84B 

-0.5 

31.4 

49.6 

70.5 

86.1 

84C 

-4.0 

21.6 

41.9 

52.5 

59.7 

64F 

0.8 

25.5 

57.3  ' 

63.9 

70.8 

91A 

-4.0 

17.6 

52.5 

76.6 

100.2 

910 

-19.6 

12.3 

42.9 

68.5 

104.8 

91E 

-6.3 

25.8 

49.3 

78.0 

93.1 

91F 

-5.6 

22.7 

42.0 

75.1 

91.7 

91G 

-13.2 

20.5 

46.4 

58.2 

79.3 

91H 

-2.8 

23.2 

45.6 

72.6 

88.4 

91 J 

7.3 

13.6 

41.9 

65.2 

72.7 

91L 

0.8 

23.6 

53.1 

73.1 

88.1 

91H 

-21.5 

23.3 

32.8 

75.7 

101.9 

91P 

-13.6 

24.5 

50.5 

61.9 

91.7 

91Q 

-18.0 

15.2 

4'  1 

77.1 

88.2 

OID 

-ii.l 

17.6 

56.1 

63.3 

84.7 

91S 

-C.5 

29.4 

53.7 

68.3 

95.6 

91T 

-1.7 

23.5 

49.0 

54.3 

65.2 

91U 

-17.5 

14.1 

48.9 

73.1 

101.9 

91V 

-13.6 

23.3 

45.3 

75.7 

101.9 

91W 

-13.2 

24.1 

46.4 

72.8 

102.1 

91Y 

-5.6 

10.7 

42.0 

70.9 

89.1 

92B 

-2.8 

29.4 

55.8 

74.8 

103.0 

92C 

4.0 

29.4 

43.5 

64.1 

83.8 

93D 

-19.0 

13.6 

50.7 

86.8 

97. S 

93E 

-4.0 

19.6 

50.7 

74.6 

A  A  At 

00.3 

93F 

0.8 

27.4 

61.7 

73.1 

101.9 

93K 

-14.8 

17.0 

48.3 

70.6 

104.5 

93J 

-17.1 

14.8 

49.6 

72.6 

105.6 

93P 

-11.3 

21.6 

54.3 

80.6 

107.5 

948 

2.7 

27.4 

63.9 

85.5 

90.8 

94F 

3.7 

32.8 

51.3 

73.5 

82.7 

95B 

-8.2 

38.6 

63.1 

84.2 

108.7 

*  Cne  of  12  KOS  assigned  actual  ratio  scales 
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flfii 

1 ft 

2£ 

*ft 

2ft 

Ift 

Q6B 

-2.9 

23.3 

49.3 

80.3 

115.2 

96D 

-8.2 

16.9 

43.7 

81.9 

99.8 

96f 

-6.3 

22.3 

52.2 

64.4 

90.4 

#W» 

96H 

-5.1 

31.4 

49.6 

77.0 

110.8 

#w»  • 

96R 

-3.2 

27.4 

57.3 

78.0 

93.5 

97B 

•19.0 

11.5 

45.4 

82.6 

107.5 

97E 

-19.6 

14.1 

38.9 

78.0 

113.8 

976 

-9.9 

28.2 

51.3 

73.5 

87 .7 

98C 

-13.6 

20.8 

48.8 

75.1 

125.3 

986 

-11.6 

22.3 

42.5 

81.5 

109.3 

98J  * 

-7.8 

24.6 

59.5 

90.0 

114.3 

*  One  of  12  MOS  assigned  actual  ratio  scales 
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Appendix  6 

INTERVIEW  PROTOCOL  FOR  SECOND-TOUR  JOB  ANALYSIS  FOR 
INFANTRYMEN  (NOS  Ill) 


JOB  ANALYSIS  INTERVIEW 


INSTRUCTIONS  TO  INTERVIEWER 

1.  The  purpose  of  this  interview  Is  to  quickly  obtain  information  about  the 
duty  positions,  activities,  time  spent  on  activities,  and  relative 
Importance  of  the  activities  for  E5  in  the  nine  Batch  A  KOS. 

2.  The  people  to  be  Interviewed  are  ESs  (or  E6s,  if  you  can't  get  ESs)  in 
the  NOS.  Try  to  get  E5s  with  36*60  months  TIS.  The  format  is  designed  for 
a  group  setting,  with  5*10  E5s. 

3.  Following  Is  a  brief  description  of  each  of  the  Items  on  the 
Interviewer's  Forms.  You  will  almost  certainly  need  to  improvise  in  order 
to  tailor  it  to  the  different  NOS.  Just  get  the  information,  and  make  sure 
that  all  of  the  Individuals  who  are  in  the  group  get  »  chance  to  be  heard, 
and  that  their  viewpoints  are  respected  and  recorded. 

4.  Specifics  about  the  five  Interview  items: 

ITEM  1:  PRIMARY  DUTY  POSITIONS 

We  compiled  a  list  of  the  authorized  duty  positions  for  ESs  from  AR  611*201. 
In  some  cases  the  setting  (eg.,  clinic  vs.  aid  station)  Is  also  listed. 

These  are  listed  on  Handout  1,  for  each  WS.  In  discussion,  find  out  if  the 
list  is  correct  and  complete.  Find  out  what  percent  of  E5s  who  are  working 
in  E5  slots  in  the  MOS  would  be  assigned  to  each  position.  Exact  percents 
aren't  necessary;  we  want  a  rough  distribution.  The  positions  with  the 
highest  concentration  are  what  we  consider  "Primary  Duty  Positions"  for  the 
Interview.  Get  the  remaining  information  on  the  highest  density  position 
first,  then  go  back  and  have  them  tell  you  how  the  information  is  different 
(or  not)  for  each  of  the  other  positions. 

ITEM  2:  NORMAL  WORK  WEEK 

Have  then  agree  on  and  describe  briefly  the  normal  work  week  (or  weeks)  for 
the  most  primary  (highest  density)  position  (and  setting,  if  necessary).  If 
the  unit  goes  through  various  cycles  such  as  post  support,  maintenance,  and 
field  cycles,  there  night  be  several  normal  weeks,  depending  on  the  cycle. 
After  you  have  the  remaining  information  on  one  normal  week,  get  it  on  the 
other  normal  weeks,  If  any,  by  discussing  how  they  differ. 

ITEM  3:  ACTIVITIES 

A  list  of  11  activities,  drawn  mostly  from  the  Gast  dimensions,  is  on 
Handout  2.  For  the  normal  week,  for  the  E5  in  this  duty  position,  find  out 
If  the  list  is  complete;  if  they  w«,nt  to  add  anything,  first  make  sure  it 
couldn't  fit  in  one  of  the  activities  listed;  then  have  them  all  add  the 
activity  on  their  lists.  If  any  activities  are  NA  for  this  position  and 
week,  everyone  puts  NA.  If  activites  are  required  for  the  position,  end  in 
the  nsrsal  week  end  setting  under  discussion,  but  are  required  only  rarely, 
find  out  how  often  and  so  note. 
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ITEM  4:  TIME  SPENT 

Ask  the*  to  note  on  their  Handout  2  the  number  of  hours  that  the  E5  Mould 
spend  on  each  activity  during  the  normal  Mork  Meek  under  discussion,  for  the 
duty  position  and  setting  under  discussion.  Instruct  the*  that  the  hours 
they  assign  should  add  up  to  40.  or  to  Mhatever  they  all  aaree  is  the 
normal.  Then  In  discussion,  and/or  by  averaging  their  assignments,  arrive 
at  a  consensus  about  the  nuaber  of  hours  spent  on  each  activity. 

ITEM  5:  IMPORTANCE 

Still  on  the  same  MQS,  E5  working  in  saw  ES  duty  position  in  same  setting, 
during  same  normal  work  week.  On  Handout  3.  have  them  add  any  activities 
that  they  all  agreed  on,  and  mark  NA  any  that  are  not  required  fm  this 
posit ion/sett ing/week.  The  question  is  “How  important  is  it  that  the  E5 
perform  each  activity  well,  in  order  that  the  unit  continue  to  function 
smoothly  and  accomplish  its  mission?*  Have  them  rank  order  the  activities 
from  1  -  Most  Important.  Discourage  ties.  Remind  them  that  they  already 
rated  time  spent,  end  importance  may  not  depend  on  time  spent;  things  that 
take  little  tioe  or  are  not  frequently  required  may  be  very  important  when 
they  are  required.  Then  in  discussion  or  by  averaging  and  then  discussing, 
arrive  at  a  consensus  rank  order.  Ties  permitted  sparingly. 

Remember  to  then  go  back  and  find  out  how  the  time  spent  and  Importance  are 
different  for  other  normal  weeks,  and  thtn  also  for  the  normal  w?.cks  i.i 
other  prinary  duty  positions.  Record  iv.  all  on  the  interview  forr^s. 
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JOB  ANALYSIS  INTERVIEW 
SKILL  LEVEL  2 

Interviewer's  For® 


HOS  Date 

Location 

Interviewer 

Interviewees: 

Paygrade: 

TIS: 

Payorade: 

TIS: 

P&ygrade: 

TIS: 

Paygrade: 

TIS: 

Paygrade: 

IIS: 

Paygrade: 

TIS: 

INTERVIEWER  *  S  FORM  1 

1.  PRIMARY  DUTY  POSITION  -  SKILL  LEVEL  2  -  11B 

Of  100  E5  in  MOS,  in  E5  positions, 
how  ®any  are  in  this  duty  position? 

Operations  Sergeant  _ 

Fire  Team  Leader  _ 

Aranun ; hum  Section  Leader  ________ 

2.  NORMAL  WORK  WEEKS  (Describe  briefly) 


INTERVIEWER’S  FORM  2 

oorr  position  discussed _ _ _ _  normal  week _ _ 

ACTIVITIES 

HOURS 

1.  Performing  MOS-specIfic  tasks  (include  exercises, 
simulations,  etc.)* 

2.  Performing  Common  Tasks  (include  exercises,  simulations,  etc.). 

3.  Training  subordinates  on  HOS-specif ic  tasks. 

4.  Training  subordinates  on  Common  Tasks. 

5.  Planning,  organizing,  and  monitoring  activities  of  subordinates. 

6.  Providing  performance  feedback  to  subordinates. 

7.  Informing  subordinates  about  plans  and  activities. 

S.  Informing  superiors  and  coordinating  with  other  units  about 
plans  and  activities. 

3.  Providing  performance  recognition  and  rewards  to  subordinates. 

10.  Counseling  subordinates  on  personal  problems. 

11.  Disciplining  or  punishing  subordinates. 

12  _ _ _ 

13.  _ _ 

14.  _ _ 

TOTAL  40 
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INTERVIEWER'S  FORM  3 
DUTY  POSITION  DISCUSSED 
ACTIVITIES 


NORMAL  WEEK 


RANK 


1.  Performing  MOS-speciflc  tasks  (include  exercises, 
simulations,  etc.)* 

2.  Performing  Cocoon  Tasks  (Include  exercises,  simulations,  etc.)* 

3.  Training  subordinates  on  MOS-specific  tasks. 

4.  Training  subordinates  on  Common  Tasks. 

5.  Planning,  organizing,  and  monitoring  activities  of  subordinates. 

6.  Providing  performance  feedback  to  subordinates. 

7.  Informing  subordinates  about  plans  and  activities. 

8.  Informing  superiors  and  coordinating  with  other  units  about 
plans  and  activities. 

9.  Providing  performance  recognition  and  rewards  to  subordinates. 

10.  Counseling  subordinates  on  personal  problems. 

11.  Disciplining  or  punishing  subordinates. 

12. 


t 


HANDOUT  1 

1.  PRIMARY  DUTY  POSITIONS  -  SKILL  LEVEL  2  -  11B 

Of  100  E5  In  MOS,  In  E5  positions, 
how  many  Are  In  this  duty  position? 

Operations  Sergeant  _ 

Fire  Tea*  Leader  _ 

Ammunition  Section  Leader  ____ 


HANDOUT  2 

HOURS 

I.  Performing  H0$»spec1f1c  tasks  (include  exercises,  _ 

simulations,  etc.)* 

2.  Performing  Common  Tasks  (include  exercises,  simulations,  etc.).  _ 

3.  Training  subordinates  on  MOS-specific  tasks.  _ 

4.  Training  subordinates  on  Coamon  Tasks.  _ 

6.  Planning,  organizing,  and  monitor iny  activities  of  sutoiul nates.  _ 

6.  Providing  performance  feedback  to  subordinates.  _ 

7.  Informing  subordinates  about  plans  and  activities.  _ 

8.  Informing  superiors,  and  coordinating  with  other  units  about 

plans  and  activities.  _ 

9.  Providing  performance  recognition  and  rewards  to  subordinates.  _ _ 

10.  Counseling  subordinates  on  personal  problems. 

II.  Disciplining  or  punishing  subordinates.  _ 

12. _ _ _ _  _ 

13.  _ _ 

14.  _ _ _ 

TOTAL  40 
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HANDOUT  3 

1.  Performing  MOS-specific  tasks  (Include  exercises, 
simulations,  etc.). 

2.  Performing  Common  Tasks  (Include  exercises,  simulations,  etc.). 

3.  Training  subordinates  on  MOS-speciflc  tasks. 

4.  Training  subordinates  on  Common  Tasks. 

5.  Planning,  organizing,  and  monitoring  activities  of  subordinates. 

6.  Providing  performance  feedback  to  subordinates. 

7.  Informing  subordinates  about  plans  and  activities. 

8.  Informing  superiors  and  coordinating  with  other  units  about 
plans  and  activities. 

9.  Providing  performance  recognition  and  rewards  to  subordinates. 

10.  Counseling  subordinates  on  personal  problems. 

11.  Disciplining  or  punishing  subordinates. 
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TASK  CLUSTE" '  ^09  SECOND  TOUR  SOLDIERS 
a  NINE  RDS 

For  each  of  the  nine  NOS,  a  sunary  of  the  changes  to  the  task  cluster 
structure  is  given,  along  with  short  descriptions  of  the  clusters. 


illui! 
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itrvwn  MS*  BflBUJ 

Of  the  12  Skill  Level  1  task  clusters,  11  were  retained  for  second 
tour;  the  cluster  on  Maintaining  and  Operating  Vehicles  was  dropped  at  the 
request  of  the  Proponent.  All  new  second  tour  tasks  were  categorized  into 
the  11  first  tour  clusters.  Therefore  no  additional  clusters  were  formed 
for  second  tour.  Although  clusters  nearly  always  changed  In  size  and  In  the 
specific  tasks  included,  the  general  content  of  ten  of  the  clusters  was 
unchanged. 

The  one  cluster  that  did  change  dramatically  was  the  cluster  entitled 
■Conduct  Tactical  Operations.*  For  second  tour,  the  cluster  has  nearly 
twice  as  many  tasks  (22  compared  to  12),  including  11  tasks  on  supervising 
and  directing  activities  of  the  fire  team,  squad,  or  platoon,  five  tasks  on 
leading  patrols  or  missions,  and  two  tasks  or  supervising  weapons  and  combat 
operations.  These  are  directly  comparable  tc  thw  two  leadership/ 
supervisory  dimensions  that  emerged  from  the  behavioral  perf.^riuince  work. 

The  46  Supervisory  Responsibilities  Questionnaire  activities  were 
categorized  Into  eight  supervision  clusters.  Ir  addition,  tt*v  domain 
Included  seven  HOS-specific  supervision  tasks  that  were  also  categorized 
Into  those  eight  clusters.  Specifically,  fotr  t*r.fs  pertaining  to  field 
reports  and  orders  were  placed  in  the  cluster  "Provide  Information, ■  and 
three  MOS-specific  training  tasks  are  in  the  cluster  "Train  and  Develop." 
Thus  these  two  clusters,  for  the  HB  £5,  represent  more  than  Just  generic 
activities  of  passing  on  information  and  providing  training. 

ihe  task  clusters  for  the  1JB  second  tour  domefn  are  listed  in  Table  2, 
along  with  brief  descriptions  of  the  cluster  content. 
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Task  Clusters:  11B  Infantryman 


FIRST  AID:  Diagnosing  administering  first  aid,  and  transporting  casualties. 


LANS  AAV  1  CAT  I  OK  AMO  RAP  READING:  Moving  ovr-  cwAnovn  terra  la,  reading  npt,  raadtag  compass, 
determining  location,  direction,  and  distance. 


NUCLEAR.  BIOLOGICAL.  CHEMICAL  (BBC):  Tasks  performed  aedar  BBC  conditions.  imludiag  Rotting 
Co  protactiva  aask  and  clothtag.  oparatiog  aad  Maintaining  BSC  equipment,  and  report  leg  BSC 
conditions. 


tfEA/OiS:  Operation,  maintenance,  and  positioning  of  wapoos  (HU  riff#.  MO  machinegue,  "AM. 
SAM,  grenades,  grenada  launcher,  ,«S  cal  pistol,  .SO  cal  mechinegun,  dragon. 


MOVEMENT  ARC  SURVIVAL  IB  FIELD:  Tasks  rs It tad  to  Battlefield  survival  In  dofontlao  tad 
•ffansiva  situations. 


COMMUNICATIONS:  Installation  and  operation  of  redio  and  field  telephone  equipment,  end 
communications  security  procedures. 


DETECT  AMD  10ERTIFY  THREATS:  Surveillance  tasks,  including  search  and  scan  procaduret,  end 
Identifying  threat  vehicles  and  aircraft. 


SICHTSICHTS:  Operation  (aovnttng,  zeroing,  engaging  targets)  and  Maintenance  of  lund-held  and 
wetpen-eounted  nlgnt  sighting  devices. 


RISES  ASD  OEKOLITICXS:  Instilling  end  disarming  etnas  and  noooy  traps,  and  Wi»u  nCi'iuy 
nonelectric  and  electric  demolition  system. 


ROVLitCHT  in  urban  TERRAIN:  Tactical  operations  in  built-up  aroas. 


CONDUCT  TACTICAL  OPERATIONS:  Supervising  snd  directing  activities  of  the  fire  team.  squad,  or 
platoon,  leading  patrol*  or  elssions.  supervising  weapons  and  cca&at  operations. 


PLAN,  ORLAmIZE,  AMO  MONITOR:  Assigning  work  tasks,  supervising  performance  of  tasks, 
conducting  inspections,  and  monitoring  equipment  condition  end  supplies. 


CLARIFY  ROtES,  i*R0nt>E  FCEODACt:  rertornencc  monitoring  any  waits* • ! ng  cf  iufcsrd 


FROVIDE  INFORMATION:  Passing  on  information  concerning  mission  and  requirements. 


RE  COG* 1 2  C ,  REMAND:  Providing  formal  and  Informal  rewards  and  recognition  for  good 
performance,  recommendinq  soldiers  for  promotion  or  awards. 


TRAIN .  DEVELOP:  Manning  and  conducting  individual  and  teen  training,  providing  career 
counseling,  end  providing  opportunities  for  leadership:. 


SUPPORT:  Listening  to  subordinates*  personal  problems,  snd  counseling,  assisting,  or 
arranging  assistance,  at  appropriate. 


DISCIPLINE,  FUR1SII:  Providing  formal  or  Informal  dltcipl irery  measures  to  subordinates. 


ACT  AS  ROOEi:  Setting  the  example  for  subordinates. 


VflM  »l 


Because  the  11  first  tour  task  clusters  could  not  adequately  absorb  all 
of  the  new  second  tour  tasks,  three  additional  clusters  were  foraed. 

Although  clusters  nearly  always  changed  in  size  and  In  the  specific  tasks 
Included,  the  general  content  of  eight  of  the  clusters  was  unchanged.  In 
three  cases,  first  tour  clusters  were  split  Into  two  »re  hoaogeneous 
clusters  by  the  addition  of  new  tasks:  Pcsltion/Lay/Fire  Howitzer  becaae 
Prepare  Position  and  Gunnery— Lay/Fire  Howitzer;  the  sines  and  daaolitlons 
tasks,  formerly  In  a  cluster  with  weapons,  foraed  a  separate  Nines/ 
Oeaolitions  cluster;  and  tasks  on  tactichi  supervisory  tasks,  forserly  in  a 
cluster  with  aoveaent  and  survival  in  the  field,  foraed  a  new  cluster. 

This  cluster.  Conduct  Tactical  Operations,  consists  of  12  tasks  on 
directing  the  cannon  crew  and  supervising  personnel  on  various  activities, 
priaarlly  aa1ntenar.ee.  Only  one  of  these  tasks  is  not  explicitly 
supervisory,  and  none  of  the  tasks  was  included  In  the  first  tour  doaaln. 

The  46  Supervisory  Responsibilities  Questionnaire  activities  were 
categorized  into  eight  supervision  clusters.  In  addition,  the  doaaln 
included  eleven  MOS-speciflc  supervision  tasks  that  were  also  categorized 
Into  those  eight  clusters.  Nine  of  these  MOS-speciflc  tasks  were 
categorized  in  the  cluster  "Train  and  Develop"  to  cover  instruction  on 
tasks  related  to  howitzer  operation.  Thus  tnis  cluster,  for  the  13B  E5, 
represents  wore  than  just  general  delivery  of  training. 

The  task  clusters  for  the  138  second  tour  doaain  are  listed  In  Table  2, 
along  with  brief  descriptions  of  the  cluster  content. 
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TABLE 


Task  Clustsr a:  13«  Coanoa  Crevaae 


FIRST  AID:  Dtagnostag  Injuries,  Mtolalsterlag  first  aid.  and  transporting  casualties. 

LAM)  RAV16AT10B  AM)  RAF  READ i IS:  Roving  ovar  wAdovb  tarrala.  raiding  saps,  reading  confess, 
determining  location,  direction.  and  dlstaece. 

W  CLEAR,  BIOLOGICAL,  CHEMICAL  (RIC):  Tasks  performed  under  BBC  conditions:  pattlao  on 
protective  mask/clothlng,  aparatiag/  malntalaing  IRC  equipment,  reporting  BBC  conditions. 

NEAPOBS:  Operation,  nalntanancs,  and  positioning  of  weapons  (MIC  rifle,  NCO  aachlaagun.  LAX. 
grenades.  grenade  launcher,  .CO  cal  occk Inague) . 

NOVEHEHT  AMO  SURVIVAL  VR  FIELD:  Tasks  ralatnd  to  battlefield  survival  in  defensive  and 
affanstve  situations. 

COMMUNICATIONS:  Installation  and  operation  of  radio  and  field  telephone  equipment,  ard 
communications  security  procaduras. 

DETECT  ARD  I0EHT1FT  THREATS:  Sunret  1  lane*  tasks,  including  search  and  tea*  procaduras,  and 
Identifying  threat  vehicles  and  aircraft. 

MIRES  ARD  DEMOLITIORS:  Installing  and  disarming  ola^t  an  booby  traps,  and  coastrwctlhv 
nonelectric  and  electric  demolition  system*. 

DRIVE:  Operating  wheeled  vehicles  and  equipment. 

NAIRTERARCE:  Maintenance  of  howitzer  and  coiqwnants  and  prise  Mover. 

PREPARE  MO  STORE  AMMUNITION:  Loading  and  transporting  aaao.  preparing  aaoo  for  firing. 

PREPARE  P0S1T10R:  Prepare  howltrar  position  and  prepare  heel  tzar  for  oparetlon. 

OJNNER r- -LAY /FIRE  HOWITZER:  lorefighting,  laying,  loading  and  firing. 

COROUCT  TACTICAL  OPERATIONS:  Supervising  and  directing  activities  of  the  cannon  erea  during 
tactical  operations. 

PLAN,  OR  LARI  ZE ,  MO  MONITOR:  Assigning  work  tasks.  Supervising  performance  of  tasks, 
conducting  inspections,  and  Monitoring  equipment  condition  and  supplies. 

ClAAlfT  ROLES.  PROVIDE  FEEDBACK :  Performance  monitoring  and  counseling  of  subordinates. 

PROVIDE  IRFORHATIOH:  Passing  on  Information  concerning  Mlsston  and  requirements. 

RECOGNIZE.  REWARD:  Providing  formal  and  Informal  rewards  and  recognition  for  good 
performance,  rtcomaending  soldiers  for  promotion  or  awards. 

TRAIN,  DEVELOP:  Planning  and  conducting  Individual  and  team  training,  providing  career 
counseling,  and  providing  opportunities  for  leadership. 

SUPPORT:  Listening  to  subordinates'  personal  problems,  and  counseling,  assisting,  or 
arranging  assistance,  as  appropriate. 

DISCIPLINE.  PUR ISA:  Providing  forma)  or  taformel  disciplinary  manure*  to  subordinates. 

ACT  AS  MODEL:  Setting  the  example  for  subordinates. 


Because  the  11  first  tour  task  clusters  could  not  adequately  absorb  all 
of  the  new  second  tour  tasks,  two  additional  clusters  were  forced.  Although 
clusters  nearly  always  changed  in  size  and  in  the  specific  tasks  included, 
the  general  content  of  nine  of  the  clusters  was  unchanged.  One  first  tour 
cluster,  Movement  and  Survival  in  Field,  was  split  into  two  sore  homogeneous 
clusters  by  the  addition  of  new  tasks:  Moveaent/Survival  In  the  Field  and 
Detect  and  Identify  Threats. 

Tne  other  new  cluster,  Conduct  Tactical  Operations,  consists  of  16 
tasks  on  controlling  the  movement  and  fire  cf  the  tank  section  or  platoon. 
Although  the  AOFP  frequency  data  Indicated  that  all  of  the  tasks  are 
perforated  by  19£  E5s,  in  fact  most  of  the  tasks  are  the  responsibility  of 
the  Piatoon  Sergeant  (E7,  skill  level  4). 

Nearly  one  third  of  the  tasks  in  the  resulting  task  domain  are  MOS- 
specific,  skill  level  1  tasks,  including  all  of  the  gunnery  tasks,  because 
of  the  Proponent's  decision  to  require  all  except  the  tank  cowanders1  tasks 
to  be  designated  for  the  skill  level  1  (El-t4)  19E. 

The  46  Supervisory  Responsibilities  Questionnaire  activities  were 
categorized  into  eight  supervision  clusters.  2n  addition,  the  domain 
included  23  MOS-specific  supervision  tasks  that  were  also  categorized  Into 
those  eight  clusters.  Three  of  these  MOS-specific  tasks  were  categorized  in 
the  cluster  “Train  tnd  Develop"  to  covet  preparation  for  and  evaluation  of 
training,  and  six  tasks  on  preparing  order's  and  reports,  were  added  to  the 
cluster  “Provide  Information."  Fourteen  of  the  MOS-specific  tasks  were 
categorized  Into  the  cluster  “Plan,  Organize,  and  Mon1tor;“  they  Include  a 
variety  of  supervision,  inspection,  and  monitoring  tasks. 

The  task  clusters  for  the  19E  second  tour  domain  are  listed  in  Table  2, 
along  with  brief  descriptions  of  the  cluster  content. 


TAILS 


Task  Clatters t  HE  Armor  Cravats 


riAST  AIO:  Diagnosing  Injuries,  administering  first  all.  ad  transporting  easaaltlas. 

LAM  NAVIGATION  AM  AAA  READING:  Roving  ow  unknown  tarrala,  ratling  aaps,  real  lag  eaapats. 
lataraialag  location,  direction,  aal  llstaaca. 

WCLEAA,  II0L0C1CAL.  CNEAICAL  (NSC)i  Tasks  performed  sadar  OC  conditions,  Including  patting 
00  protective  aask  and  clothing,  operating  and  mlataintng  IIC  oqatpaeat.  tad  reporting  NSC 
conditions;  also  iaclodet  preparing  tAo  taak  for  MC  conditions. 

NEAPOtS:  Operation,  aalatonanco,  aad  positioning  of  weapons  (All  rifle,  AM  mchtnogwn,  LAM. 
grenades,  grenade  laancAer,  AS  snbaacAlnagea,  .49  cal  pistol). 

MOVEMENT  AM  SURVIVAL  II  FIELD:  Tasks  rotated  to  fcatttaflold  servlval  In  defensive  aad 
effoastve  sltaatlons,  lacladlag  tank  positions. 

COMMUNICATIONS:  Installation  aad  operation  of  radio,  field  tolapAone.  aad  latorcciml  cat  Ions 
egelpaeat.  and  coinlcatlons  socef  tty  procodares. 

DETECT  AM  IOCITIFT  TMtEATS:  Surveillance  tasks,  lacladlag  soarck  and  seta  procodares.  aad 
Identifying  tAreat  vehicles  aad  aircraft. 

AIKS  AM  OCAOLITIMS:  Install  lag  and  dl tanning  nines  and  booty  traps,  and  caastracting 
nonelectric  and  electric  dowel Itioa  system. 

PREPARE  TASK  AM  TASK  SYSTEMS  FOR  OPCIATIOIS:  Aalntonance  aad  servicing  of  tank  aatoaottve 
system  (aeapon  system  excledod),  stoning  aaminltlon,  preparing  aad  secarlng  stations. 

OPERATE  TAW  (EXCEPT  MEAPON  SYSTEMS):  Driving  and  recovering  tank,  operating  non-weapon 
♦  cenponents,  performing  daring  and  after  operations  chocks. 

PREPARE  TAW  NEAPOR  SYSTEMS  FOR  OPERATION:  Maintaining  and  boresighting  min  gen  and 
mchlneguns,  performing  firing  cheats. 

OPERATE  TAM  NEAPORS  SYSTEMS:  loading  and  aaloadlng  guns,  engaging  targets. 

CCMUCT  TACTICAL  OPERATIORS:  Supervising  and  directing  activities  of  the  tank  crew,  squad,  or 
platoon  during  tactical  operations. 

PLAI.  ORGANIZE,  AM  MONITOR:  Assigning  wort  tasks,  supervising  performance  of  tasks, 
conducting  Inspections,  and  mnttorlng  equipment  condition  and  supplies. 

CLARIFY  ROLES.  PROVIDE  FEEDRACX:  Performance  mnttorlng  and  counseling  of  subordinates. 

PROVIDE  INFORMATION:  Passing  on  information  concerning  mission  and  requirements. 

K COGNIZE,  REVARO:  Providing  formal  and  Informal  rewards  and  recognition  for  good 
per  form  nee,  recommending  soldiers  for  promt  ton  or  awards. 

TRAIN,  DEVELOP:  Planning  and  conducting  Individual  and  team  training,  providing  career 
counseling,  and  providing  opportunities  for  leadership. 

SUPPORT:  Listening  to  subordinates'  personal  problem,  and  counseling,  assisting,  or 
arranging  assistance,  as  appropriate. 

DISCIPLINE,  PUNISH:  Providing  formal  or  Inform!  disciplinary  measures  to  subordinates. 

ACT  AS  MOOEL:  Sotting  the  example  for  subordinates. 
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Of  the  nine  first  tour  31C  clusters,  five  were  retained  for  second 
tour.  Although  these  clusters  changed  somewhat  in  size  and  specific  tasks 
included,  the  general  content  between  first  and  second  tour  regained 
consistent.  One  first  tour  cluster,  "Customs  and  Laws  of  War,"  was  dropped 
because  only  one  of  the  tasks  included  in  the  cluster  appeared  in  the  second 
tour  domain. 


The  first  tour  clusters  could  not  adequately  absorb  all  of  the  new 
tasks  in  the  second  tour  dooain.  Hence,  three  additional  clusters  were 
foroed.  Specifically,  the  first  tour  cluster,  BCoaoun1cations  Procedures,* 
was  subdivided  into  two  clusters  titled  "Communication"  and 
"Administration."  This  was  done  to  accomodate  23  new  adainistratlve 
functions.  Similarly,  the  first  tour  cluster  "Radios"  was  subdivided  Into 
"Radio  Sets”  and  "Teletypewriters*  to  accommodate  several  new  radio  and 
teletypewriter  related  tasks.  Finally,  the  first  tour  cluster  titled 
"Combat  Procedures"  was  subdivided  into  two  common  skills  clusters, 
"Move/Survive  in  the  Field"  and  "Detect  and  Identify  Threats,"  again  to 
accommodate  several  new  tasks.  A  second  reason  for  creating  these  two 
clusters  was  so  that  similar  comon  task  clusters  would  appear  across  all 
nine  MOS. 


In  all,  11  nonsupervlsory  clusters  were  used  to  describe  the  second 
tour  31C  Job.  Six  of  these  are  common  skills  task  clusters  and  five  are 
MOS-specific  technical  task  clusters.  The  reason  the  31C  have  only  six 
common  skills  clusters  as  opposed  to  seven  is  because  the  "Communication" 
cluster  (which  Is  a  common  task  cluster  in  the  other  eight  MOSs)  contains 
mostly  job-specific  tasks  for  this  MOS. 

The  46  Supervisory  Responsibility  Questionnaire  (SRQ)  items  were 
categorized  Into  eight  supervision  clusters  proposed  by  Yukl.  The  domain 
contained  12  additional  supervision  tasks  that  were  also  categorized  into 
these  eight  clusters.  While  most  of  the  12  additional  tasks  reflected  more 
general  Army-wide  supervisory  responsibilities  and  were  similar  in  content 
to  SRQ  items,  two  of  the  tasks  were  very  specific  to  the  31C  second  tour 
job.  These  were  supervising  subordinates  in  the  installation,  grounding,  or 
removal  of  communications  equipment  and  Issuing  Instructions  for 
installation  of  radio  teletype  equipment.  These  MOS-specific  aspects  of 
supervision  also  emerged  from  the  critical  incidents  work  and  are  contained 
in  the  MOS-specific  behavioral  dimension  titled  "Managing  the  RATT  Rig." 
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Task  Clnst* 


31C  Single  Channel  Radio  Operator 


FUST  AID:  Task*  that  relate  to  diagnosing  Injuries,  adainlsttr'ng  first  aid.  ead 
transporting  casualties. 

LANO  NAVIGATION  A I©  AW  READING:  Tashs  related  to  moving  o»»r  unknown  tertafn.  reeding  maps, 
reading  a  coapess,  determining  location,  direction,  and  distance. 

NUCLEAR.  BIOLOGICAL,  CHEMICAL  (SBC):  Tasks  performed  under  FIC  conditions.  Including  gutting 
on  protective  east  end  clothing,  operating  and  aaintalnlng  a»C  equipment,  and  reporting  BBC 
conditions. 

WEAPONS:  Includes  teaks  on  operation,  aaintenanca.  and  positioning  of  NIB  rtflo.  MO 
eecMtnogun,  LAM,  mints  and  booby  traps,  and  grenades. 

MOVEMENT  AMO  SuavlVM.  1«  FIELD:  Tasks  related  to  battlafield  survival  In  defensive  end 
offensive  posit  loot, 

COMMUNICATIONS:  Tasks  related  to  preparing  and  sanding  radio  messages,  eperet',:^  radio  sots, 
and  installing  entennes  end  other  commintesttan*  related  equipment. 

• 

DETECT  AND  IDENTIFY  THREATS:  Covers  surveillance  tasks,  IncluS:"^  ;«*rch  and  scan  procedures, 
and  identifying  threat  vehicles  and  aircraft. 

A&sist STRATI ve  FUNCTIONS:  Involves  tasks  such  as  preparing  fores  and  reports,  conduct i*v 
waclpeant  Inventories,  and  or  tting/re^  isvlng/ravising  SOPs. 

GENERATORS:  Tasks  Involving  tnstal ling/operating  geit'.rafe-i  t .  troubleshooting  generators,  and 
pat  forming  PMCS  or.  generators. 

RADIO  SETS:  Includes  tasks  on  Installing  radio  sett.  pyrfcralng  troubleshooting  procedures, 
and  crr,d„:t  »“CS  on  radio  sets. 

TELETYPEWRITERS:  Tasks  related  to  installing  and  operating  teletypewriters,  troubleshooting 
teletypewriter  sets,  end  conducting  PMCS  on  teletypewriters. 

PLAN.  ORGANIZE.  MONITOR:  Tesks  re  1* ted  to  assigning  work  tasks,  supervising  performance  of 
tasks,  conducting  Inspections,  end  Monitoring  equipment  condition  end  supplies. 

CLARIFY  ROLES,  PROVIDE  FEECbACk:  Covers  performance  Monitoring  and  counseling  of 
suberd  twites. 

PROVIDE  INFORMATION:  Include.,  tasks  related  to  passing  on  Information  concerning  the  mission 
and  requirements. 

RECOGNIZE.  REWARD:  Providing  formal  end  informal  rewards  and  raccgnitton  for  good 
performance,  retnemending  soldiers  for  promotion  or  awards. 

TRAlN/DEVCLOr:  Planning  and  conducting  Individual  and  taam  training,  providing  career 
counseling,  and  providing  opportunities  for  leadership. 

SUPPORT:  L(*ter>lng  to  subordinates'  personal  problems,  and  counseling,  assisting,  or 
arranging  assistance,  as  appropriate. 

DISCIPLINE.  PUNISH:  Providing  formal  or  Informal  disciplinary  measures  to  subordinates. 

ACT  AS  MODEL:  Setting  the  eseaple  for  subordinates. 


fi3B  Light  Wheel  Vehicle  Mechanic  Ta$K  PttMlfl 

Of  the  12  first  tour  63B  clusters,  seven  remained  virtually  Identical 
In  content  for  second  tour,  although  there  were  some  changes  in  cluster  sue 
iie<£  specific  tasks  Included. 

Because  the  first  tour  clusters  could  not  adequately  absorb  all  of  the 
new  tasks  ir*  the  second  tour  domain,  several  additional  cluster j  "®re 
forced.  The  first  tour  cluster,  “Movement  In  the  Field,  was  subdivided 
into  two  second  tour  clusters,  "Hove/ Survive  in  the  Field  and 
"Conauni cation.”  This  was  done  in  order  to  accowodate  several  new 
cowaunlcations  and  move/survive  tasks  thtt  appeared  in  the  second  tour 
domain.  SlMilarly,  the  first  tour  "Comoat  Procedures”  cluster  was 
subdivided  Into  ’Detect  and  Identify  Threats*  and  'Land  Navi gat i on/Hap 
Reading  "  again  in  order  to  accommodate  several  new  tasks.  A  second  reason 
for  creating  the  clusters  described  above  was  that  so  similar  cowaon  task 
clusters  would  appear  across  all  nine  MOS. 


For  the  MOS-specific  clusters,  the  first  tour  cluster  ‘General 
Maintenance”  was  subdivided  into  'Administration*  and  “Preventive/General 
Maintenance.-  This  subdivision  was  done  to  accommodate  20  new 
administrative  tasks  (the  first  tour  domain  contained  only  tow 
administrative  tasks  which  were  in  the  "General  Maintenance"  cluster).  An 
entirely  new  cluster,  'Generators,  Small  Engines,  and  Equipment,"  was  also 
created.  The  tasks  in  this  cluster  were  new  additions  to  the  second  tour 
domain.  Finally,  two  first  tour  clusters,  "Brakes'  and  'Steer ing/Suspens ion 
$v$ terns  -  were  coined  into  one  cluster  for  second  tour.  The  main  reason 
for  this  combination  was  that  each  cluster  contained  relatively  rew  casts , 
and  these  systems  require  similar  troubleshooting  approaches.  ™rther^ 
since  we  wanted  to  keep  reasonable  the  tota1  number  of  clusters  from  which 
SMEs  would  be  asked  to  select  tasks  in  a  subsequent  step,  collapsing  these 
two  clusters  seemed  appropriate. 


In  all,  a  total  of  15  nonsupervisory  clusters  were  used  to  describe  the 
second  tour  31  job.  Seven  of  these  are  common  skills  clusters  and  the 
remaining  eight  are  MOS-specific  technical  task  clusters. 


The  46  Supervisory  Responsibility  Questionnaire  (SRQ)  Items  were 

1  Th<*  rinmin 

categorized  iniu  eiyin.  iwn  kimm*  K.  «v 

contained  50  additional  supervision  tasks  that  were  also  categorized  into 
these  eight  clusters.  Eighteen  of  these  were  Army-wide  supervisory 
activities  and  were  similar  in  content  to  SRQ  Items.  The  remaining 
additional  supervisory  tasks  reflected  MOS-specific  aspects  of  supervision 
and  fell  into  the  'Plan,  Organize,  Monitor”  cluster.  These  tasks  Included 
established  maintenance  priorities,  determining  corrective  action  for 
maintenance  problems,  and  supervising  maintenance  on  brake  systems,  cooling 
systems ,  engines,  exhaust  systems,  transmissions,  steering  assemblies,  and 
other  vehicle  related  parts/systems.  Many  of  these  aspects  of  supervision 
also  emerged  from  the  critical  incidents  work  and  are  contained  in  the  MOS- 
specific  behavioral  dimension  titled  'Checking  Repairs  Made  by  Other 
Mechanics.- 
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TABLE 


Tnk  CliiUru  lil  Light  Khee!  Vehicle  KrcknH  Task  Clusters 


fit  ST  AID:  Tasks  that  relate  to  dt agios  leg  Injuries,  administering  first  aid,  eel 
transporting  casualties. 

LAW  RAY I GAT I on  ABO  RAT  READ I *6:  Tasks  related  to  —vlng  over  unknown  tarrala,  reading  um^s . 
reading  a  compass,  daurwinimg  location.  dlrectioa.  aad  distance. 

BOCLCAA.  BIOLOGICAL,  CNEMCAi  (RtC):  Tasks  performed  under  iEC  conditions,  Including  patting 
an  protective  mask  and  clothing,  oparatiag  and  ■aiataiatng  ISC  equipment,  and  reporttag  BBC 
conditions. 

REArORS:  Includes  tasks  on  operation,  maintenance,  aad  positioning  of  Bit  rifle,  MO 
■adilnagun,  LAN,  altias  end  booby  traps,  aad  grenades. 

novEKHT  ABO  SURVIVAL  IB  FIELD:  Tasks  ralatad  to  Battlefield  survival  la  dafanslvo  aad 
#f  fans  Wo  posit  tons. 

COMHUfllCATIOBS:  Tasks  ralatad  to  preparing  and  sandlag  radio  messages,  oparatiag  radio  tats, 
aad  Installing  antennas  and  other  coa—n  lest  toss  ralatad  equipment, 

DETECT  AKO  IDERTIFY  THREATS:  Covers  surveillance  tasks,  Inciting  starch  aad  scan  procedcres. 
and  tdantlfylng  throat  vehicles  aad  aircraft. 

ADHiniSTRATICd:  Involves  task*  such  as  reviewing  maintenance  requests,  Batarlal  conditio?) 
status  reports,  operator's  qualifications  records,  ate.;  conducting  equipment  'nvaotorics,  and 
wrltlng/revlewlng/revlilng  SOP*. 

GEREkAYORS,  TRAILERS.  SHALL  ENSUES.  ABO  EOUIPREBT:  Tasks  iRvoWIno  adjusting,  replacing. 
Inspecting,  tasting,  aw)  troubleshooting  generators,  trailers,  smU  anginas,  and  othar 
equipment. 


PRE  VERT  I V  t  /  SERtRAi  m  l  a  i  tNAsCi  ■  i  nt  tuoai  itii>  ui>  ■  1  v "  P?».f  cr  '  ng 

maintenance  on  vahiclos. 


BRAKES,  STEERIHG.  SuSPEeSlON  SYSTEMS:  Tasks  related  to  adjusting,  Inspecting,  replacing, 
repairing,  and  troubleshooting  brakes,  steering,  and  suspension  syttsss. 

ELECTRICAL  SYSTEMS.  TEST  SYSTEMS:  Contains  tasks  that  Involve  adjusting,  inspecting, 
replacing,  repairing,  and  troubleshooting  electrical  end  test  system. 

VEHICLE  RECOVERY  SYSTEMS;  Includts  tasks  related  to  recovering  vehicles  as  well  as 
Inspecting,  servicing,  and  adjusting  viMclo  recovery  equipment. 


POWER  TRAIN,  CLUTCH,  ERCIKE  SYSTEMS:  Tasks  related  to  adjusting.  Inspecting,  replacing, 
repairing,  and  troubleshooting  power  treln,  clutch,  and  engine  systems. 


fiitc,  COOL  ini .  LuGeiCATiOS.  EKnAUST  ST5TE“S:  Covers  tssEs  IrvcW  adjusting.  inspect  i*»0; 
replacing,  repairing,  and  troubleshooting  fuel,  cooling,  lubrication,  and  exhaust  syttaea. 


plan.  Oft CAR  I  IE,  HORItOG:  Tasks  ralatad  to  assigning  work  tasks,  supervising  performance  of 
tasks,  conducting  Inspect  lone,  end  monitoring  equipment  condition  and  supplies. 


CLARIFY  ROLES.  PROVIDE  FEEDBACK:  Covers  performance  monitoring  end  counseling  o' 
subordinates. 


PROVIDE  IRfORMATIGS:  Include*  tasks  related  to  passing  on  Information  concern Irg  the  mission 
and  requirements. 

BECOGRIZE.  REWARD:  Providing  formal  end  Informal  rewards  and  recognition  for  good 
performance,  re  co-ending  soldiers  for  promotion  or  awards. 


TRAli/DEVEtOP:  Planning  and  conducting  Individual  and  teas  training,  providing  career 
counseling,  and  providing  opportunities  for  leadership. 


SUPPORT:  Listening  to  subordinates'  personal  problems,  and  counseling,  assisting,  or 
arranging  assistance,  as  appropriate. 


OiSCIPLIHE,  PUalSH:  Providing  formal  or  Informal  disciplinary  measures  to  subordinates. 
ACT  AS  MOOCL:  Setting  the  example  for  subordinates. 
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Of  the  nine  first  tour  71L  clusters,  eight  were  retained  for  second 
four.  Although  these  clusters  changed  somewhat  In  size  end  specific  tasks 
Included,  the  general  content  was  consistent  between  first  end  second  tour. 

One  first  four  cluster,  'Field  Techniques'  was  subdivided  Into  three 
cowan  skills  clusters  as  follows:  'Communication, "  "Movement/Survival  In 
the  Field,"  and  "Detect  end  Identify  Threats.*  This  subdivision  was  deemed 
necessary  because  there  were  several  new  task  statements  added  for  second 
tour  that  could  not  be  adequately  absorbed  by  the  single  first  tour  cluster. 
In  addition,  these  particular  clusters  were  used  so  that  similar  common  task 
clusters  would  appear  across  all  nine  MQS.  In  all,  a  total  of  11 
nonsupervisory  clusters,  seven  cf  which  were  common  skills  clusters  and  four 
of  which  were  MOS-specif ic  clusters,  were  used  to  describe  the  second  tour 
711  Job. 

The  46  Supervisory  Responsibility  Questionnaire  (SRQ)  items  were 
categorized  into  eight  supervision  clusters  proposed  by  Yukl.  The  domain 
contained  15  additional  supervision  tasks  that  we^e  also  categorized  into 
these  eight  clusters.  The  majority  of  these  15  tasks  reflected  “Plan, 
Organize,  Monitor*  and  "Traln/Develop"  activities.  If  should  be  noted, 
however,  that  the  15  additional  tasks  generally  reflected  common  Army-wide 
supervisory  responsibilities  rather  than  MOS-specIfic  components  of 
supervision.  Also,  the  content  of  most  of  these  tasks  was  highly 
overlapping  with  SRQ  items. 


TABLE  Task  Clusters:  71L  Administrative  Specialist 


FUST  AID:  Tasks  that  relate  to  diagnosing  injuries.  administering  first  aid.  and 
transporting  casualties. 

LAdO  AAV  I  Cat  I  On  AXO  NAP  READIES:  Tasks  relatad  to  moving  over  unknown  terrain,  reading  snips, 
reading  a  coapett,  determining  location,  direction,  and  distance. 

MCLEAN,  BI0L0C1CAI'  CHEMICAL  (BBC):  Tasks  performed  under  NBC  conditions.  Including  putting 
on  protective  cask  and  clothing,  operating  and  maintaining  UC  equipment,  and  reporting  MC 
conditions. 

WEAPONS:  Includes  tests  or.  operation,  aelntonaaca.  and  positioning  of  Nil  rifle.  MO 
mechtnegun,  law.  sines  and  booby  traps,  and  grenades. 

NOvEnENT  AND  SURVIVAL  II  FIELD:  Tasks  related  to  battlefield  survival  t>  defensive  end 
offensive  positions. 

COMMUNICATIONS:  Tssks  related  to  preparing  and  sending  radio  Besieges,  operating  radio  sets, 
end  instslling  antennas  and  otber  coeumn lent  Ions  related  equipment. 

OETECT  AND  IDENTIFY  THREATS:  Covort  serve  1  lienee  tasks,  including  searca  and  scan  procedures, 
end  identifying  threat  vehicles  end  aircraft. 

PREPARE  AND  MAINTAIN  FRES/FORMS:  Involves  tails  such  at  identifying  publications  requiring 
changes  and  updating  thea.  establishing  and  reviewing  files,  end  preparing  end  reviewing 
forms/records. 

CORRESPONDENCE:  Involves  typing,  proofreading,  and  editing  doct*entt,  asset* ling 
correspondence,  dispatching  documents,  and  routing  Inc  suing  distribution. 

CLASSIFIED  MATERIAL:  Tasks  dealing  with  securing  end  Maintaining  classified  materials  as  well 

as  identify ii>u  reportir^,  a.~  ccrroctihc  security  violations. 

OFFICE  ADMINISTRATIVE  FUNCTIONS:  Includes  planning  office  layout,  aslntelning  office 
resource? ,  selecting  details,  end  wr Itlng/revtewing/revtsing  unit  SCP. 

PLAN.  OR  CAR  I ZE,  MONITOR*  Tasks  related  to  assigning  work  tasks,  supervising  perfomar.ee  of 
tasks,  conducting  inspections,  end  aonitorlng  equipment  condition  and  supplies. 

CLARIFY  ROLES,  PROVIDE  FEEDBACK:  Covers  perforasnee  aonitorlng  end  counseling  of 
Subordinates. 

FROVIOt  INFORMATION:  Includes  tasks  related  to  posting  on  (nfonaation  concerning  the  ■lesion 
and  requiremntt. 

RECOtNIZE.  REHARD:  Providing  formal  and  infomal  revardt  and  recognition  for  good 
performance,  recosE»~51*v;  soldiers  for  promotion  or  awards. 

TRAIR/DEVELOP:  Planning  and  conducting  individual  and  teas  training,  providing  carter 
counseling,  and  providing  opportunities  for  leadership. 

SUPPORT:  Listening  to  subordinates'  personal  problem,  and  counseling,  assisting,  or 
arranging  assistance,  es  appropriate. 

DISCIPLINE.  PUNISH:  Providing  form'  or  tofomel  disciplinary  Measures  to  subordinates. 

ACT  AS  MOOEL:  Setting  the  example  for  subordinates. 
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Although  the  12  first  tour  task  clusters  represented  aost  of  the  new 
second  tour  tasks,  the  cluster  structures  were  realigned  In  soae  cases  to 
fora  aore  hoaogeneous  clusters .  The  general  content  of  nine  of  the  clusters 
was  unchanged.  Three  first  tour  clusters,  each  of  which  Included  tasks  on 
aoveaent  and  survival  In  the  field  as  well  as  tasks  on  cooaunlcatlons  and 
detecting  threats,  were  reformed  as  Koveaent/Survlval  In  the  Field, 
Coaaunlcatlons,  and  Detect  and  Identify  Threats.  Two  clusters  of  NBC 
related  tasks  were  combined. 

One  new  cluster.  Conduct  Tactical  Operations,  was  foraed  to  represent 
tactical  supervisory  responsibilities.  It  consists  of  only  three  tasks,  on 
the  designation  and  construction  of  positions.  None  of  the  tasks  was 
Included  In  the  first  tour  doaaln. 

The  88M  seco-d  tour  task  doaaln  ha*  only  22  tasks  froa  the  MOS 
Soldier's  Manual,  and  21  froa  the  MOS-specific  AOSP,  coapared  with  103 
Coamon  Tasks.  Additionally,  the  8£M  Soldier's  Manual  does  not  distinguish 
between  skill  levels  1  and  2;  a  coablned  task  list  Is  presented. 

The  46  Supervisory  Responsibilities  Questionnaire  activities  were 
categorized  Into  eight  supervision  clurters.  In  addition,  the  doaaln 
Included  five  MOS-specific  supervision  tasks  that  were  also  categorized  Into 
those  eight  clusters.  One  of  these  MOS-specific  tasks  was  categorized  in 
the  cluster  ‘Train  and  Develop*  to  cover  training  in  loading/unloading 
procedures,  and  four  tasks  on  vehicle  inspections,  aotor  pool  operations, 
and  fire/safety  pregras  were  added  to  the  cluster  "Plan.  Organize,  and 
Monitor. * 

The  task  clusters  for  the  88M  second  tour  domain  are  listed  in  Table  2, 
along  with  brief  descriptions  of  the  cluster  content. 
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TAILS  Tail  Clusters:  88M  Motor  Transport  Oporotor 

FUST  AID:  Diagnosing  injuries,  administering  first  aid,  and  transporting  casualties. 

LAND  NAVI  CAT  I  Ok  A AC  NAP  READING:  Noting  over  unknown  terrain,  reading  amps,  reading  compass, 
determining  location,  direction,  and  distance. 

MCLEAN.  IIOLOCICAL,  CNI A 1  CAL  (NIC):  Tests  performed  under  NIC  conditions,  including  putting 
ee  protective  desk  and  clothing,  operating  and  maintaining  ISC  equipment,  and  reporting  R8C 
conditions;  also  Includes  operating  tracts  coder  ISC  conditions. 

WEAPONS:  Operation,  maintenance,  and  positioning  of  neapeas  (NIC  rtfla,  MO  nachinegen,  LAM. 
grenades,  grenade  launcher,  n‘nes,  booby  traps). 

MOVEMENT  An  SURVIVAL  It  FIELO:  Tasks  related  to  battitflald  survival  ta  defensive  end 
offensive  situations,  including  defensive  driving  procedures  under  aabush  or  attack. 

COMMUNICATIONS:  Installation  and  operation  of  radio  and  communications  security  procedures. 

DETECT  AMO  IDENTIFY  TNI  EATS:  Surveillance  tasks,  including  search  and  scar,  procedures,  and 
identifying  threat  vehicles  and  aircraft. 

OPERATE  VEHICLES:  Driving  under  various  conditions,  coupling/uncoupltng.  loading  and 
transporting  cargo  and  personnel,  end  parking. 

FILL  OUT  FOAMS:  Completion  of  operator.  Occident,  and  dispatch  ferae. 

PHCS/PREPARE  FOR  MOVEMENT:  Tasks  related  to  vehicle  eointenance  end  aoveaent. 

RECOVERY :  Self -recovery  of  vehicles  and  recovery  by  other  vehicles. 

CONDUCT  TACTICAL  OPERATIONS:  Supervising  end  directing  activities  in  preparing  positions. 


CLARIFY  ROLES,  PROVIDE  FEEOSACX:  Performance  monitoring  and  counseling  of  subordinates. 

PROVIDE  INFORMATION:  Pas&l'ig  on  information  concerning  mission  end  requirements. 

RECOGNIZE,  REWARD:  Providing  formal  ard  informal  rewards  and  r«coqr.1t1o.i  for  good 
performance,  recommending  soldiers  for  promotion  or  awards. 

TRAIN,  DEVELOP:  Planning  and  conducting  Individual  and  teas  training,  providing  career 
counseling,  and  providing  opportunities  for  leadership. 

SUPPORT:  Listening  to  subordinates'  personal  problems,  and  counseling,  assisting,  or 
arranging  assistance,  at  appropriate. 

DISCIPLINE,  PUNISH:  Providing  formal  or  informal  disciplinary  measures  to  subordinates. 
ACT  AS  NOOEL:  Setting  the  exa^le  for  subordinates. 
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Of  the  10  first  tour  91A  clusters,  seven  regained  virtually  Identical 
In  content  for  second  tour,  although  there  were  sow  changes  In  cluster  size 
and  specific  tasks  Included.  The  "Vehicles"  cluster  f row  first  tour  was 
dropped  because  only  three  of  the  tasks  froot  this  cluster  appeared  In  the 
second  tour  dooain.  These  three  tasks  were  recategorlzed  Into  the 
"Prepare/Halntaln  Medical  Facilities  and  Equipment1  cluster. 

Because  the  first  tour  clusters  could  not  adequately  absorb  all  of  the 
new  tasks  in  the  second  tour  dowln,  three  additional  clusters  were  forwd. 
The  first  tour  cluster  "Hoveaent/Survlval  In  the  Field"  contained  two 
comuni  cat  Ion*  tasks.  In  the  second  tour  dowln,  however,  there  were 
several  additional  cowunlcatlons  tasks.  Hence,  a  new  cluster, 
"Ccmunlcations"  was  forwd.  Beyond  accomodating  the  additional 
comunlcatlons  tasks,  another  reason  for  breaking  these  Iteas  out  and 
foralng  a  new  cluster  was  so  that  the  sane  cowon  task  clusters  would  appear 
across  all  nine  HOS.  The  first  tour  cluste*  "Patient  Care  and  Treatment" 
was  subdivided  into  two  task  cluster  titled  "Perfora  Medical  Tests  and 
Procedures"  and  "Paf.lent/Casualty  Care."  This  was  done  In  order  to  absorb 
17  new  tasks  that  appeared  In  the  second  tour  dowln.  Similarly ,  the  first 
tour  cluster  “First  Aid"  was  subdivided  into  "First  Aid"  and  "Bandages, 
Splints,  and  Dressings*  again  to  accomodate  several  new  tasks  relevant  to 
these  performance  areas. 

In  all,  12  nonsupervisory  clusters  were  used  to  describe  the  second 
tour  91A/B  job.  Seven  of  these  are  comon  skills  task  clusters  and  five  are 
MOS-specific  technical  task  clusters. 

The  46  Supervisory  Responsibilities  Questionnaire  iteas  were 
categorized  into  eight  supervision  clusters  proposed  by  Yukl.  The  doaaln 
contained  21  additional  supervision  tasks  that  were  also  categorized  into 
these  eight  clusters.  All  but  one  of  these  19  tasks  reflected  more  general 
Army-wide  supervisory  responsibilities  and  were  overlapping  siailar  in 
content  with  SRQ  Iteas.  The  one  supervisory  task  that  was  specific  to  the 
91A/B  job  was  supervise  aedical  operations. 
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TABLE 


Tank  Clatters:  tlA/B  Medical  Specialist 


FIRST  AID:  Tasks  that  relate  to  dlagnoatag  Injuries.  administering  first  aid.  end 
transporting  casual tits. 

LAMO  NAVIGATION  AID  NAP  READING:  Tasks  ralatod  to  moving  ovur  unknown  terrain,  reading  aaps. 
reading  a  coopast,  detaretntng  location,  direction,  and  litttnce. 

b»  CLEAN.  IHX. OC I CAL ,  CHEMICAL  (ISC):  Tasks  performed  nndor  ISC  conditions,  tnclndlno  petting 
on  protective  east  and  clothing,  operating  and  ■alntatotag  atC  equipment.  and  reporting  ItC 
conditions. 

WEAPONS:  Includes  tasks  on  operation,  aatntenaaca,  and  positioning  of  R18  rifle,  MO 
anchlnegun,  LAW,  nines  and  booh/  traps,  and  grenades. 

MOVEMENT  aid  SURVIVAL  II  FIELD:  Tasks  ralatod  to  battlefield  survival  In  defensive  and 
•ffanslva  positions. 

COMMUNICATIONS:  Tnsks  related  to  preparing  end  sanding  radio  nes sages,  operating  radio  seta, 
and  installing  antennas  and  other  coaaunt cat  ions  related  equipment. 

DETECT  AM)  IDENTIFY  TWtEATS:  Covers  tar  vail  lance  talks,  including  anarch  and  scan  procedures, 
end  identifying  threat  vehicles  and  aircraft. 

BAaOACES.  SPLIITS,  AN  DRESSINGS:  Tasks  Involving  dressing  noendt.  applying  bandages,  and 
tnaoblllzlng  broken  bones. 

PERFORM  NEDICAL  TESTS  ADO  PROCEDURES:  Includes  tasks  dealing  with  administering  Injections, 
neeturing/recordlng  patients'  vital  signs,  and  performing  various  tests  and  aedical 
procedures. 

PREPARE /NAI NT All  NEDICAL  FACILITIES  AID  EQUIPMENT:  Tasks  related  to  sterilizing  equipment, 
inspecting  and  ealntainlng  aedical  faculties,  end  preparing  for  pattenta  to  receive 
area  turn. 

ADMINISTRATION:  Covers  drafting,  receiving,  for  filing  aedical  charts,  lab  reports,  end 
records  as  well  as  requesting,  receiving  and  controlling  aedical  supplies  and  equipment. 

PAT i ENT /CASUALTY  CARE:  Assisting  patients  with  personal  hygiene,  attending  to  casualties,  and 
briefing,  receiving,  and  escorting  patients. 

PLAN,  ORGANIZE.  MONITOR:  Tasks  related  to  assigning  work  tasks,  supervising  performance  of 
tasks,  conducting  Inspections .  and  monitoring  equipment  condition  and  supplies. 

CLARiFT  ROLES,  PROVIDE  FEEDBACK:  Covers  performance  monitoring  and  counseling  of 
subordinates. 

PROVIDE  INFORMATION:  Includes  tasks  related  to  passing  on  tnforaatlon  concerning  the  atsston 


RECOGNIZE.  REWARD:  Providing  formal  and  Informal  rewards  and  recognition  far  good 
performance,  recoamanding  soldiers  for  promotion  or  awards. 

TRAIN/DEVElOP:  Planning  and  conducting  Individual  and  teaa  training,  providing  careor 
counseling,  and  providing  opportunities  for  leadership. 

SUPPORT:  Listening  to  subordinates'  personal  problems,  and  counseling,  assisting,  or 
arranging  assistance,  as  appropriate. 

DISCIPLINE ,  PUNISH:  Providing  formal  or  Infernal  disciplinary  Matures  to  subordinates. 
ACT  AS  MODEL:  Setting  the  example  for  subordinates. 
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The  95B  domain  changed  dramatically  since  the  first  tour  job  analyses 
and  criterion  development  effort.  The  second  tour  domain  contained  over 
twice  as  many  tasks  as  the  first  tour  domain,  and  many  of  these  represented 
new  activities/ job  requirements.  The  primary  orientation  of  the  958  job  is 
changing  from  garrison  police  activities  to  a  more  strictly  Infantry 
orientation.  Because  this  change  Is  not  yet  complete,  first  and  second  tour 
958s  are  responsible  for  both  major  types  of  job  tasks.  In  fact,  956s 
currently  have  four  "missions*  as  follows:  the  two  described  above, 
battlefield  circulation  control,  and  processing  eneqy  prisoners  cf  war. 

Of  the  12  first  tour  95B  clusters,  the  content  of  seven  was  virtually 
identical  for  second  tour.  However,  several  new  tasks  were  added  to  all  of 
these  clusters.  Items  from  the  first  tour  cluster  "Movement/ Survival  in  the 
Field"  were  recategorized  into  two  clusters,  one  of  which  ("Hovement/Control 
of  Personnel")  was  first  tour  cluster  and  the  other  of  which  ("Contact  with 
Hostile  Personnel";  was  a  new  second  tour  cluster.  This  was  done  to 
accommodate  36  new  tasks  that  dealt  with  these  content  areas.  The  first 
tour  "Field  Techniques"  cluster  was  subdivided  into  "Fighting  Positions"  and 
"Oetect  and  Identity  Threats, 14  again  to  accommodate  33  new  tasks  that 
appeared  in  the  second  tour  domain. 

Because  the  first  tour  clusters  could  not  adequately  absorb  all  of  the 
tasks  in  the  second  tour  domain,  two  new  clusters  were  formed  ("Security" 
and  "Administration").  Also,  three  relatively  small  first  tour  clusters 
("Respond  to  Crimes."  "Make  Apprehensions,"  and  "Investigate  Crimes")  were 
collapsed  into  one  cluster  and  then  subdivided  Into  two  clusters  titled 
•Activities  Related  to  Crimes"  and  "Traffic  Related  Tasks."  This  was  done 
to  accommodate  19  new  traffic  tasks  and  30  new  crime  tasks  that  appeared  in 
the  second  tour  domain. 

In  all,  14  nonsupervisory  task  clusters  were  used  to  describe  the 
second  tour  95B  job.  Seven  of  these  are  common  skills  clusters  and  seven 
are  MQS-specific  technical  clusters. 

The  46  Supervisory  Responsibility  Questionnaire  items  were  categorized 
into  eight  supervision  clusters  proposed  by  Yukl.  The  domain  contained  34 
additional  supervision  tasks  that  were  also  categorized  into  these  eight 
clusters.  Approximately  20  of  these  35  tasks  reflected  more  general  Army* 
wide  supervisory  responsibilities  and  were  similar  in  content  to  SKQ  items. 
The  remaining  additional  supervisory  tasks  were  specific  to  the  95B  job  and 
fell  into  two  clusters:  "Plan,  Organize,  Monitor*  and  "Train/Develop."  The 
MOS-specific  supervision  tasks  Included  activities  such  as 
planning/supervising  patrols,  organizing  squads,  supervising  security  force 
operations,  preparing  oral  squad  operations  orders  and  fragmentary  orders, 
and  establisning/supervising  crime  prevention  programs.  Most  of  these  MOS- 
specific  aspects  of  supervision  also  emerged  from  the  critical  incidents 
work  and  are  contained  in  the  MOS-specific  behavioral  dimensions  titled 
"Leading  the  Team  in  a  Tactical  Environment." 


TAILS  Tell  Claitars:  III  Military  Tolic* 


fUST  A10:  Teikt  that  relate  to  diagnosing  Injuria*.  administering  f*r»t  all.  **d 
transporting  casualties. 


tjUfO  NAVIGATION  AMO  MAP  READING:  Tail*  reUted  to  moving  over  unfcaown  tarrala,  reading  aapt. 
rmsding  a  compel*.  determining  location,  HrwtlM,  end  iiatanco. 


conditions. 


WEAPONS:  Include*  tail*  on  operation,  malntenenco,  aad  positioning  of  H16  rifla.  RW 
■ccMivtflun,  LA#,  mints  and  booby  trap*,  aad  franaoai. 

movement  AMD  SURVIVAL  11  TlELOt  Tail*  rail  tad  to  battlefield  survival  tn  defensive  and 
offensive  pot  It  loot. 

COnniRlCATIOflS:  Tail*  related  to  preparing  and  tending  radio  aatiagoi.  npa^atlag  radio  tott. 
pod  installing  antonnai  and  otbar  communications  related  equipment. 

DETECT  ARO  IDENTIFY  THREATS:  Ccvers  surveillanc*  task*.  Including  toarck  and  acan  procedures. 
»!  1  identifying  throat  vehicles  and  aircraft. 

HOVEHEBT/CONTROl  Of  PERSOHNEL:  Involvos  moveMht  of  personnel  andar  co«Dat  and  noncortMt 
conditions  to  Include  processing  and  supervising  the  security  cf  EPw/CI. 


VEHICLE  OPERATION  AMO  MAINTENANCE:  Tatkl  rolatad  to  drivtng,  recovering,  and  maintaining 
military  vehicles. 


CONTACT  with  HOSTILE  PERSONNEL:  Includes  tails  dealing  with  Individual  and  teen  battle 
techniques  as  noil  at  civilian  control  Matures. 


A.-riviTtrc  art  me  to  piinfS:  Tasti  related  to  Interacting  with  ctIm  suspects,  collecting 
and  handling  evidence,  and  responding  to  criminal  activities. 


ADMINISTRATIVE:  Contains  a  variety  of  adminlatrat  v#  functions  such  as  accounting  for  police 
activities,  selecting  details,  preparing/** intalnln-  .»ats,  and  wrlttng/revlewing/revlsing 
SOP. 


SECURITY:  Tasks  Involving  Mlntalning  the  accourtetillty/securlty  of  classified  docuMnts  and 
securing  sites  containing  classified  Mtarlal. 

TRAFFIC  RELATED  TASKS:  Covers  tesks  relating  to  covering  treffte  flow,  enforcing  traffic 
regulations,  ond  investigating  traffic  accidents. 

PUN.  ORGANIZE.  MONITOR:  Tasks  ralatad  to  assigning  work  tasks,  supervising  performance  cf 
tasks,  conducting  inspections,  and  monitoring  aquIpMnt  condition  and  supplies. 

CLARIFY  HOLES.  PROVIDE  FEEDBACK:  Covers  performance  monitoring  end  stniiisc'iinu  Of 
subordinates. 


PROVIDE  INFORMATION:  Includes  tasks  ralatad  to  passing  on  taformatton  concerning  the  mission 
aad  requirements. 

RECOGNIZE.  REVARD:  Providing  formal  and  Informal  rewards  aad  recognition  for  good 
per  forma  net .  recomaendlag  soldiers  for  promotion  or  awards. 


TAAIi/DEYELOP:  Planning  and  conducting  Individual  and  tees  training,  providing  career 
counseling,  and  providing  opportunities  for  leadership. 

SUPP0S1:  listening  to  subordinates'  personal  problems,  and  counseling,  assisting,  or 
arranging  assistance,  as  appropriate. 

OISCIPLIME.  PUNISH:  Providing  formal  or  Informal  disciplinary  measures  to  subordinates. 


ACT  AS  MODEL :  Setting  the  asaapla  for  subordinates. 
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Appendix  X 


NQS-SPECIFIC  PERFORMANCE  DIMENSIONS 
FOR  SECOND  TOUR  SOLDIERS 
IN  NINE  NOS 


For  each  of  the  nine  MOS,  a  sunary  of  the  major  changes  to  the 
MOS-specif ic  performance  dimensions  Is  given,  followed  by  a  list 
of  the  dimensions  and  rating  scales. 
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The  content  of  six  of  the  12  first  tour  performance  categories  was 
unchanged  for  second  tour.  Five  categories  were  modified  to  reflect 
additional  performance  requirements/expectations  (such  as  maintaining 
equipment  even  when  not  specifically  told  to  do  so.  being  able  to 
demonstrate  use  of  weapons  In  addlf on  to  being  able  to  use  the  weapons 
expertly,  and  supervising  subordinates  In  guard  positions). 

The  greatest  difference  between  first  and  second  tour  responsibilities  Is 
evidenced  by  the  critical  incidents  pertaining  to  leadership  responsibili¬ 
ties,  The  first  tour  category  Assisting  and  Leading  Others"  was  divided 
into  two  second  tour  categories:  "Leading  the  Team”  and  "Supervising 
Soldiers  in  the  Field."  These  two  categories  include  responsibilities  for 
ensuring  that  troops  have  the  required  supplies  and  equipment,  ehsuring  the 
safety  and  well-being  of  soldiers,  briefing  troops  about  the  mission, 
ensuring  that  work  is  properly  completed,  using  sound  judgment  to  accomplish 
the  mission,  and  leading  by  example. 

The  first  and  second  tour  performance  category  names  are  shown  below;  the 
second  tour  performance  category  definitions  are  listed  on  the  next  page. 


FIRST  TOUR  PERFORMANCE  CATEGORIES  SECOND  TOUR  PERFORMANCE  CATEGORIES 

Maintaining  Supplies,  Equipment,  Maintaining  and  Accounting  for 
and  Weapons  Weapons  and  Equipment 

Assisting  and  Leading  Others  Supervising  Soldiers  In  the  Field 

Leading  the  Team 

Navigation  Navigation 

Use  of  Weapons  and  Other  Use  of  Organic  Weapons  and  Equipment 

Equipment 

Fie  id  Sanitation,  Personal  Field  Sanitation,  Personal  Hygiene 

Hygiene,  and  Personal  Safety  and  Personal  Safety 

Fighting  Positions  Fighting  Positions 

Avoiding  Enemy  Detection  Avoiding  Enemy  Detection 

Operating  a  Field  Phone/Radio  Operating  a  Radio  Set 

Reconnaissance  and  Patrol  Reconnaissance 

Guard  and  Security  Outies  Guard  and  Security  Duties 

Prisoners  of  War  Prisoners  of  War 

Courage  and  Proficiency  in  Proficiency  in  Battle 

Battle 


SECOND  TOON  NOS  PERFORMANCE  CATEGORY  NAMES  AND  DEFINITIONS  FOR  NOS: 

INFANTRYMAN  (110) 


A.  Ralntatnlng'tnd  Accounting  for  Equipment  tod  Weapons 

Ham  effective  It  each  soldier  U  ensuring  that  ell  equipment  end  weapons  an  woll 
maintained  and  available  for  nto  in  the  field? 

I.  Supervising  Soldiers  le  the  Field 

Row  effective  Is  each  soldier  In  entering  the  troops  have  necessary  supplies/ 
equipment  And  ensuring  the  safety  and  Mll-bethg  of  soldier? 

C.  Loading  the  Ton 

Row  effective  Is  each  soldier  when  loading  a  toan  In  a  field  envlronnot? 

0.  Navigation 

Row  effective  is  each  soldier  la  using  sc  laatloaal  equipment  end  navigating  In 
the  field? 

I.  Rte  of  Organic  Weapons  and  Equipment 

Row  effective  is  each  soldier  In  using  organic  Mapons  and  equlpaent  safely  and 
proficiently? 

F.  Field  Sanitation,  Personal  Hygiene,  and  Personal  Safety 

Now  effective  It  each  soldier  In  Maintaining  sanitary  conditions,  personal 
hygiene,  end  personal  safety  In  the  field? 

S.  Fighting  Positions 

Now  effective  is  each  soldier  In  preparing  a  fighting  position,  range  cords,  and 
sector  ske.-het? 

N.  Avoiding  Enemy  Detection 

Now  effective  Is  each  soldier  In  avoiding  eneay  detection  during  movement  and  In 
established  defensive  positions  while  In  the  field? 

I.  Operating  a  Radio  Set 

How  effective  is  each  soldier  In  putting  e  radio  Into  operation  end  using  It 
properly? 

J.  Reconnaissance 

Now  effective  Is  each  soldier  in  performing  reconnaissance  activities? 

K.  Cuarri  and  Security  Outlet 

Now  effective  it  each  soldier  in  performing  sergeant  of  the  guard  and  security 
duties  end  sanntng  observation  posts? 

L.  Prisoners  of  War 

Now  affective  Is  each  soldier  la  guarding  and  processing  prisoners  of  war  during 
field  exercises  or  In  coafcat? 

N.  Proficiency  In  Battle 

How  effective  It  each  soldier  in  demonstrating  proficiency  in  engaging  the  enemy 
during  field  exercises  or  in  combat? 


1-3 


13B  Cannon  Crewmen  Behavioral  Performance  Categories 

All  ten  of  the  first  tour  performance  categories  were  retained  for 
second  tour.  No  explicitly  supervisory/ leadership  performance  categories 
were  added  to  represent  the  138  second  tour  job.  Although  most  of  the  138 
E5s  are  gunners  who  rarely  load  or  handle  aaminition,  set  up  communications, 
load  or  unload  the  howitzer,  receive  or  relay  communications,  or  keep 
records,  the  categories  have  been  retained  for  those  soldiers  who  are  not 

fit  gunners.  The  content  of  five  categories  was  unchanged  for  second  tour. 

ive  categories  were  modified  to  reflect  additional  performance 
requirements/expectations  (such  as  ensuring  equipment  Is  loaded  and 
operational  prior  to  field  missions,  and  knowing  the  status  of  repair  parts 
and  following  through  to  get  them  to  the  unit). 

The  first  and  second  tour  performance  category  names  are  shown  below; 
the  second  tour  performance  category  definitions  are  listed  on  the  next 
page. 


FIRST  TOUR  PERFORMANCE  CATEGORIES 

SECOND  TOUR  PERFORMANCE  CATEGORIES 

Loading  Out  Equipment 

Loading  Out  Equipment 

Driving  and  Maintaining  Vehicles, 
Howitzers,  and  Equipment 

Driving  and  Maintaining  Vehicles, 
Howitzers,  and  Equipment 

1 i an^pur  1 1  iiu/SOi  1 1  ity/StOr  {  ny  anu 
Preparing  Ammunition  for  Fire 

Transport  1  ng/Scrt i r.g/Stcr  1  ng  and 
Preparing  Ammunition  for  Fire 

Preparing  for  Occupation/Emplacing 
Howitzer 

Preparing  for  Occupation/Emplacing 
Howitzer 

Setting  Up  Communications 

Setting  Up  Communications 

Gunnery 

Gunnery 

Loading/Unloading  Howitzer 

Loading/Unloading  Howitzer 

Receiving  and  Relaying 
Communications 

Receiving  and  Relaying 

Communications 

Recording /Record  Keeping 

Recording/Record  Keeping 

Position  Improvement 

Position  Improvement 

1-4 


SECOND  TOUR  NOS  PERFORMANCE  CATEGORY  RAKES  AND  DEFINITIONS  FOR  NOS: 

CANNON  CREMNAN  (138) 


A.  leading  Out  CqalpMUt 

Non  effective  It  Mch  ioMIit  It  drawing.  Inspecting,  lending,  Mi  ncowrln 
egetpmat  needed  for  field  alsstoes? 


•.'Driving  tad  Maintaining  Vehicles,  Rovltzer,  tad  Cgolpamrt 

low  effective  It  each  toldlor  It  cooductltg  fotpoctlott  ttd  mlatenance  chocks; 
ensuring  that  rootloo  torvlco  ttd  repairs  tro  ptrfonttd;  ensuring  that  vehicles 
tad  howitzers  art  aparatad  properly? 


C.  Trans portlag/Sertlng/Stor lag  tad  Preparing  amenities  for  Ptra 

Row  effective  It  tack  toldlor  It  loading.  t taring,  tad  steering  aaaetlttaa  on 
vehicles  far  trattport;  organizing  aad  pro  tact  lag  aaaa  la  field;  proper  lag  fens, 
charges,  aad  project  Hat  for  fire? 

0.  Preparing  far  Occapatlao/faplaciag  doeltzar 

Row  affective  It  etc*  soldier  la  salectlag  location  aad  approach  roots  for 
heel tzar;  at lag  band  aad  are  tlgaalt  to  felde  bool tzar;  steer lag  batoplato  aad 


K.  Satttag  dp  Coanualcattaas 

Row  affective  It  each  toldlor  la  laving,  beryl eg.  and  staking  coomnlcatlons  elre; 
connecting  and  chocking  radio  telephones  far  operation;  troubleshooting 
caoaealcatloh  problem  aad  asking  repairs? 


F.  ■  Connery 

Mao  affective  It  oach  toldlor  la  directing  the  aaplacemnt  ef  the  col  line  tor  and 
alalng  posts;  obtaining  tight  picture:  sotting  deflection,  gvadraat.  aad  site  to 
crest;  boresighting? 


C.  loadtng/Daloadlng  booltzer 

loo  affective  Is  each  soldier  In  swabbing  and  chocking  bore  for  obstructions; 
loading  projectile  and  charge,  handling  aaaunttloa  and  loadar/raaaar? 


I.  deceiving  and  delaying  Corneal cat lens 

loo  affective  It  oach  soldier  la  aoaltorlng  radio  telephone  for  coamndt; 
receiving  coamndt  froa  dTO  or  ether  creo  neuters;  orally  relaying  coamndt? 


1.  Record tng/decord  Keeping 

■ow  effective  Is  each  soldier  la  keeping  gunner  reference  cards,  range  cards, 
end  records  ef  fire? 


J.  Position  fnprovomnt 

New  effective  Is  each  soldier  In  ensuring  that  fox  boles  are  dug  properly, 
caaouflage  net  Is  erected  and  positioned  effectively,  and  gun  position  Is 
mlntalnod  ia  an  orderly  mnaer? 
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All  eight  of  the  first  tour  performance  categories  were  retained  for 
second  tour.  The  content  of  one  category  was  unchanged,  while  the  other 
seven  categories  were  modified  to  reflect  additional  performance 
reoulreaents/expectatlons  (such  as  ensuring  that  weapons  are  loaded  Quickly 
and  accurately  rather  than  actually  loading,  perfo.-alng  prepare  to  fire 
checks,  anticipating  Maintenance  needs). 

One  leadership  dimension  entitled  'Assuming  Supervisory  Responsi- 
billtles  In  Absence  of  Tank  Commander'  was  added  to  cover  performing 
supervisory  functions  and  controlling  aoveaent  of  the  tank  when  the  tank 
coonander  1$  absent. 

The  first  and  second  tour  performance  category  names  are  shown  below; 
the  second  tour  performance  category  definitions  are  listed  on  the  next 
page. 


FIRST  TOUR  PERFORMANCE  CATEGORIES 

Maintaining  Tank,  Tank  Systems, 
and  Associated  Equipment 

Driving/Recovering  Tanks 

Stowing  Ammunition  Aboard  Tanks 

Loading/Unloading  Guns 

Maintaining  Guns 

Engaging  Targets  with  Tank  Guns 

Operating  Communication  Equipment 
Preparing  Tank  for  Field  Problems 


SECOND  TOUR  PERFORMANCE  CATEGORIES 

Maintaining  Tank,  Tank  Systems, 
and  Associated  Equipment 

Driviny/Recoverlng  Tanks 

Stowing  Ammunition  Aboard  Tanks 

Loading/Unloading  Weapons 

Maintaining  Weapons 

Engaging  Targets  with  Tank  Weapon 
Systems 

Operating  Communication  Equipment 

Preparing  Tank  for  Field  Problems 

Assuming  Supervisory  Duties  in  Absence 
of  Tank  Commander 
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SECOND  TOUR  NOS  PERFORMANCE  CATEGORY  NAMES  AND  DEFINITIONS  FOR  NOS 

TANK  CREWMAN  (19£> 


A.  Maintaining  Tank,  Tank  Systems  and  Associated  Equipment 

Mo«  affactlva  Is  each  soldier  In  Inspecting,  claanlng.  servicing,  and  performing 
Minor  repairs  on  tank  hull,  suspension  system,  ana  tank  parts  (a. 9.,  Oat tar  las. 
turret/ftra  control  systaa,  and  associated  aqulpaant)  and  In  performing  propara- 
to-flre  checks? 


I.  Driving/Recovering  Tasks 

Mow  affective  Is  each  soldier  In  safoly  and  efficiently  operating,  Maneuvering, 
positioning,  and  recovering  tanks? 


C.  Sieving  Ammunition  Aboard  Tanks 

Hov  effective  Is  each  soldier  to  sorting,  stowing,  and  securing  aaaunttion 
aboard  tanks;  preparing  and  Maintaining  samunttlon? 


D.  Loading/Unloading  Weapons 

Hov  effective  Is  each  soldier  In  loading  rounds  fo,‘  aatn  gun.  loading/unloading 
Machine  guns? 


E.  Maintaining  Weapons 

Hov  effective  is  each  soldier  In  cleaning.  Inspecting  and  perfonalng  Minor 
repairs  on  weapon  and  veepon  co^>onents  (e.g.,  Main  gun  breech  block  Assembly)? 

f.  Engaging  Targets  with  Tank  Weapon  System 

M<*»  effective  is  each  soldier  In  identifying  targets,  boraslghting  and 
calibrating  weapons;  operating  main  gun  and  fire  controls;  adjusting  firv  aini 
firing  on  targets  IAW  coamands;  preparing  and  using  ran  •  cards? 


C.  Operating  Comuntcetior.  Equipment 

Hov  affective  Is  each  soldier  in  following  proper  communications  procedures 
and  acting  at  a  radio  Monitor? 


H.  Preparing  Tanks  for  Field  Problems 

Hov  affective  is  each  soldier  in  uploading  tanks  vith  equipment;  preparing 
tanks  for  special  operations  or  conditions  (e.g..  rail  loading,  nuclear 
attack,  ate.)? 


I.  Assuming  Supervisory  Outtes  in  Absence  of  Tank  Commander 

Kov  affactlva  Is  each  soldier  in  performing  supervisory  functions  end 
controlling  movement  of  vahicla  when  TC  Is  absent? 
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All  six  first  tour  performance  categories  Mere  retained  for  second 
tour.  However,  It  was  necessary  to  modify  five  of  these  to  reflect 
additional  second  tour  performance  requireeents/expectatfons  (e.g.,  ensuring 
that  equipment  Is  serviced  and  parts  replenished,  enforcing  safety  rules t 
ensuring  that  messages  are  sent  and  received.  Inspecting  logs).  The  sixth 
category,  "Providing  Safe  Transportation,"  contains  half  of  Its  original 
content  covering  reading  maps  and  driving  safely.  The  content  relatino  to 
packing  and  preparing  for  movement  was  extracted  and  used  In  a  new  scale, 
^Preparing  for  Movement." 

One  MOS-specific  supervisory  category,  "Managing  the  RATT  Rig,"  was 
added  for  second  tour.  This  dimension  contains  supervisory  behaviors  that 
are  required  of  second  tour  soldiers  In  the  position  of  team  chief. 

The  first  and  second  tour  performance  category  names  are  shown  below; 
the  second  tour  performance  category  definitions  are  listed  on  the  next 
page. 


FIRST  TOUR  PERFORMANCE  CATEGORIES 
Inspect/Service  Equipment 
Installing  Equipment 
Operate  Coanunication  Devices 
Preparing  Reports 
Maintaining  Security 
Providing  Safe  Transportation 


SECOND  TOUR  PERFORMANCE  CATEGORIES 
Inspect/Service  Equipment 
Installing  Equipment 
Operation  Cocauni cat ions  Devices 
Preparing  Reports 
Maintaining  Security 
Providing  Safe  Transportation 
Preparing  for  Movement 
Managing  the  RATT  R.ig 
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SECOND  TOUR  NOS  PERFORMANCE  CATEGORY  NAMES  AND  DEFINniONS  FOR  NOS 
SINGLE  CHANNEL  RADIO  OPERATOR  (31C) 


A.  !  aspect  lag  end  Servicing  Equi  parent 

Kov  effective  It  each  soldier  in  inspecting  equipsant,  trouoleshootlng  problem, 
palling  preventive  Maintenance,  end  ensuring  that  equipment  I*  serviced? 


I.  Installing  Eqetpennt 

No*  effective  it  each  soldier  in  installing  cqaipaent  and  asking  it  reedy  For 
operation? 


C.  Operating  Co—unicationa  Devices 

Ion  effective  it  each  soldier  in  operating  coanun lections  devices  ltd  providing 
for  an  accurate  and  tiaely  flee  of  information? 


0.  Preparing  Reports 

doe  effective  it  each  soldier  in  preparing  reports,  filing  form ,  inspecting 
logs,  end  recording  incidents  occurring  unite  en  shift? 


C.  nalntstnlng  Security 

Now  effectivo  it  each  soldier  In  aaintatntng  security  of  equipment  and 
information? 


F.  Preparing  for  Ncvemnt 

Ho*  effective  it  each  soldier  in  preparing  for  the  transport  of  equlpaent  to 
sits:? 


C.  Providing  Safa  Transportation 

Ho*  effective  is  each  soldier  in  locating  sites  end  providing  safe  transport 
of  equipment? 


H.  Henaging  RATI  Rig 

Ho*  affective  it  each  soldier  in  aonltorlng  and  Maintaining  the  overall 
effectiveness  of  the  RATT  rig  equlpaent  end  teen? 


63B  Light  Wheel  Vehicle  Mechanic  Behavioral  Performance  CsteuorJfii 

For  four  of  the  li  first  tour  performance  categories,  there  **rt  no 
changes  for  second  tour.  Six  categories  were  codified  to  reflect  additional 
second  tour  performance  requirements/expectations  (e.g.,  conducting 
Inventories,  accounting  for  tools/test  equipment,  ensuring  that  soldiers 
follow  unit  SOP,  always  following  appropriate  troubleshooting  procedures, 
proficiently  diagnosing  malfunctions,  suggesting  core  efficient  ways  to 
accomplish  work).  One  category,  “Vehicle  Operation,*  was  renamed  ‘Equipment 
Operation"  since  mechanics  operate  equipment  other  than  vehicles  (e.g., 
generators). 

The  HOS-specIfic  supervisory  responsibilities  required  for  second  tour 
63B  soldiers  were  captured  by  adding  the  dimension  "Checking  Repairs  Made  by 
Other  Mechanics,*  which  includes  checking  repairs  to  ensure  that  they  were 
made  correctly. 

The  first  and  second  tour  performance  category  names  are  shown  below; 
the  second  tour  performance  category  definitions  are  listed  on  the  next 
page. 


FIRST  TOUR  PERFORMANCE  CATEGORIES 
Inspect/Test  Equipment  Problems 
Troubleshooting 

Performing  Routine  Maintenance 
Repair 

Use  Toois/Test  Equipment 
Using  Technical  References 
Vehicle  Operation 
Safety  Mindedness 
Administrative  Duties 
Recovery 

Determine  Task  Requirements 


SECOND  TOUR  PERFORMANCE  CATEGORIES 
Inspect/Test  Equipment  Problems 
Troubleshooting 

Performing  Preventive  Maintenance 
Repair 

Use/Account  for  Tools/Test  Equipment 

Using  Technical  References 

Equipment  Operation 

Safety  Mindedness 

Administrative  Duties 

Recovery 

Determine  Task  Requirements 
Check  Repairs  Made  by  Others 
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SECOND  TOUR  NOS  PERFORMANCE  CATEGORY  NAMES  AS  9CFXRITXOXS  FOR  NOS: 
LIGHT  WHEEL  VEHICLE  MECHANIC  (OB) 


A.  Inspecting  tad  Testing  Cqalpant  Trek  laws 

Row  affective  Is  sack  Midler  la  Inspecting  tad  testing  egatpasat 
aalfeactteoa'f 

I.  Checking  repairs  Node  by  Othar  Nacbaates 

Mow  effective  Is  aach  sold  far  la  checking  repairs  sada  by  atbar 
ascbaaics  tad  eesurtag  that  repairs  art  aade  correctly! 

C.  Trowhleshoetlag 

Raw  affactlva  Is  tact  so  Id  tor  la  daterotatag  tba  caasat  af  ogatpaaat 
■alfeaet toast 

•.  far  fora  lag  fravaattva  Re  Intone ace  Oacti  tad  Services 

Rea  affactlva  Is  aacb  soldier  la  carrying  oat  scbadakd  aslattataca  tasks 
ta  keep  vat 1c las  speratteaelf 

C.  Repair 

Rea  affactlva  Is  tack  soldier  la  correcting  atlfaacttaas  ta  aaka  vakfclas 
operational! 

F.  Rstag/Accewattag  far  Teals  tad  Test  Cgvtpaeet 

Haa  affactlva  is  sack  soldier  la  salacttag,  as  lag  aslatafatag  tad 
tccaaattag  far  tools  tad  aqolpaantT 

t.  Is  tag  Tackatcal  koferaacos 

Row  affactlva  ts  tack  soldier  ta  locating  sad  astag  tackatcal  dacaaeats 
(e.g.,  TRs.  LOt.  otc.)  akaa  porforatag  toast 

R.  Cgatpaent  Operation 

Rom  affactlva  ts  tack  soldier  ta  operating  tad  socarlag  egalpaeatt 

I.  Safety  Rlndednets 

Row  off  vet  1*o  Is  each  soldier  ta  knowing  and  following  safety  precaatteas 
tad  entering  that  other  Midlers  fallow  Mfaty  precoettees? 

J.  Administrative  Oatlos 

Raw  affective  Is  aach  Midtar  la  ceaplettag  paperwork  and  asking  disposition 
af  paperwork? 

I.  Dottrel sing  Task  RaqvtraaatiU 

Raw  affective  Is  uck  Midtar  ta  acgalrtng  necessary  astartals  before  bag  t  no  tag 
Utkst 

l>  Recovery 

Row  affactlva  ts  sack  soldier  ta  doteretotag  c^atpaant  sad  wthods  for 
recovering  dltaklad  vahtclesT 


Ml 


2U_AAain1i_trative  Specialist  Behavioral  ^ertorasncr  Categories 

One  first  tour  performance  category,  "Posting  Regulations"  was  deleted 
because  SHFs  felt  that  It  was  not  sufficiently  Important  to  warrant  Its 
Inclusion  as  a  separate  dimension.  Three  of  the  first  four  categories  were 
unchanged  for  second  tour.  The  remaining  four  first  tour  categories  were 
revised  to  reflect  Increased  requirements /performance  expectations  required 
of  second  tour  soldiers  (e.g.,  prioritizing  own  work,  wl lllngness/ablllty  to 
use  word  processing  equipment,  ensuring  that  office  eoulpment  Is  properly 
maintained).  The  first  tour  category,  "Keeping  Records"  was  changed  to 
"Correspondence  Management-  because  SCs  felt  that  the  new  title  would  be 
more  appropriate  for  the  content  of  the  dimension.  Finally,  the  filing 
system  used  by  the  Army  was  changed  since  development  of  the  first  tour 
scales.  The  old  filing  system  (TAFFS)  was  replaced  by  MARKS.  Ho  H05~ 
specific  supervisory  dimensions  were  added  for  the  71L  dob. 

The  first  and  second  tour  performance  category  names  are  shown  below; 
the  second  tour  performance  category  definitions  are  listed  on  the  next 
page. 


FIRST  TOUR  PERFORMANCE  CATEGORIES 

Prepare,  Type,  Proofread  Documents 

Distribute/Dispatch  Documents 

Maintain  Office  Resources 

Posting  Regulations 

Establish/Maintain  Hies  IAW 
TAFFS 

Keeping  Records 

Safeguard  Classified  Material 

Provide  Customer  Service 


SECOND  TOUR  PERFORMANCE  CATEGORIES 
Prepare,  Type,  Proofread  Documents 
Process  and  Distribute  Documents 
Maintain  Office  Resources 

EstablHh/Naintain  Files  IAW  MARKS 
Correspondence  Management 

w  f  .  .  j  M  ...  1/1.  J  »*  _ . _ _  .  .  i 

Mlicyuoi'u  Liaas'i  icu  naici  iai 

Provide  Customer  Service 
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SECOND  TOO*  KOS  PERFORMANCE  CATEGORY  NAMES  AMO  DEFINITIONS  FOR  NOS 

AONINI STRATI YE  SPECIALIST  (711) 


A.  Preparing,  Typing,  and  Proofreading  Oocmnnts 

Now  at fact l v«  ft  each  soldier  In  preparing  documnts  to  mot  scheduled 
deadlines;  using  the  proper  form  and  foreatt;  ensuring  that  coop  to  ted 
coplaa  contain  no  errors? 


t.  Free  at  sing  and  Districting  Docanaants 

Ire  affoctlva  Is  each  soldlar  in  precasting  and  distributing  documnts  ia  a 
tlmly  Hiwr;  routing  documnts  according  to  tba  comand  distribution  plan; 
and  sorting  incouing  documnts  proparly 7 


C.  Maintaining  Office  ftasourcas 

Ire  effective  Is  each  soldlar  in  onsurtng  that  off  lea  ogutpmnt.  supplies,  and 
publications  are  proparly  mlntalnod? 

0.  Establishing  and/or  Maintaining  files  IAN  MAMS 

lire  affective  Is  aach  soldlar  ia  using  KMUCS  to  ensure  that  all  office 
documnts  ore  properly  Maintained? 


E.  Correspondence  Mntgemnt 

lire  affective  is  aach  soldier  la  hooping  accurate  suspense  logs  and 
Maintaining  correspondence  accountability? 

f,  Preparing  and  Safeguarding  Classified  Materials 

Are  affective  Is  each  scldtar  In  preparing,  handling,  storing,  end  destroying 
classified  Mtarlsls  In  accordance  with  Array  regulations? 


C.  Providing  Custocer  Service 

Now  affective  Is  aach  soldier  in  providing  asaful  assistance  to  alt  custcaars. 
both  in  the  office  and  over  the  telephone? 
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fiSMJtetar  Transport  Operator  Behavioral-Performance  Cavgoorlei 

hine  of  the  ten  first  tour  performance  categories  were  retained  for 
second  tour.  The  content  of  four  categories  was  unchanged,  while  fire 
categories  wen  modified  to  reflect  additional  performance 
requirements/expectations  (such  as  higher  levels  of  driving  proficiency, 
ensuring  that  maintenance  has  been  performed  before  allowing  vehicles  to 
leave  tne  motor  pool,  responsibilities  for  meeting  cornel tments  on  time). 

One  first  tour  category,  entitled  'Loading  Cargo  and  Transporting 
Personnel ,■  was  replaced  by  two  categories,  covering  loading  and 
transporting  cargo  separately  from  loading  and  transporting  personnel.  No 
MOS-speciflc  supervisory  category  was  added,  as  the  behavioral  examples 
suggested  that  the  Job  is  more  technical  than  supervisory. 

The  first  and  second  tour  performance  category  names  are  shown  below; 
the  second  tour  performance  category  definitions  are  listed  on  the  next 
page. 


SECOND  TOUR  PERFORMANCE  CATEGORIES 
Driving  Vehicles 
Vehicle  Coupling 

Checking  and  Maintaining  Vehicles 
Using  Maps/Following  Proper  Routes 
Loading  and  Transporting  Cargo 
Loading  and  Transporting  Personnel 
Parking  and  Securing  Vehicles 
Performing  Administrative  Duties 
Self -Recovering  Vehicles 
Safcty-Mindedness 
Performing  Dispatcher  Duties 


FIRST  TOUR  PERFORMANCE  CATEGORIES 
Driving  Vehicles 

VehiCl®  f'nimHnn 

Checking  and  Maintaining  Vehicles 

Using  Maps/Fol lowing  Proper  Routes 

Loading  Cargo  and  Transporting 
Personnel 

Parking  and  Securing  Vehicles 
Performing  Administrative  Duties 
Self -Recovering  Vehicles 
Safety-Mindedness 
Performing  Dispatcher  Duties 
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SECOND  TOUR  NOS  PERFORMANCE  CATEGORY  NAMES  AND  DEFXBTIQNS  FOR  NOS 
MOTOR  TRANSPORT  OPERATOR  (OR) 


A.  Driving  Nklclu 

lev  effective  Is  each  soldier  le  epersflne  hruy  vesicles  (e.g..  tracks.  Jeeps, 
tractors,  end  sovitratlers)  le  a  safe,  effective,  and  levf#1  Manor  under  verlees 
conditions? 


B.  Vehicle  Coupling 

lev  effective  Is  each  soldier  le  ceap11ng/uacoupl1n|  tracks,  tractors,  eed 
trailers  eccerdleg  to  standard  operating  procedures? 


C.  Check 1>t  end  Nslstalnlnt  Vehicles 

lev  effective  Is  each  soldier  In  serf  ere  leg  PNCS;  check  leg  vehicles  far  problems 
before,  derlaf.  and  after  eoent facets;  recognizing  vehicle  prehlaea  eed  taking 
appropriate  actleaf 


•.  Vs lag  Naps /felloe leg  Proper -teetas 

lev  effective  Is  each  soldier  le  secarlag  proper  asps  es  needed;,  becoming 
foal  liar  with  rentes  ahead  ef  tiae  nhen  appropriate;  eslag  nee  effectively; 
fellevlng  prescribed  rentes;  and  arriving  at  canHtaaets  ae  tint 


K.  leading  end  Transporting  Cargo 

■on  effective  Is  each  soldier  In  supervising  the  leading  ef  cargo;  checking  that 
cargo  Is  properly  distributed,  secured,  and  blocked;  failenlng  special 
Instruct loos  nhen  hauling  dangerous  or  hazardous  cargo? 


f.  leading  and  Transporting  Personnel 

■on  effective  Is  each  soldier  la  folloolag  appropriate  pi  scederas  nhen 
transporting  personnel? 


I.  Parking  and  Securing  Vehicles 

Ion  effective  Is  each  soldier  In  setting  the  brakes  and  transmission  properly 
nhen  parking  vehicles;  securing  vehicles  nhen  they  are  net  la  operation? 


I.  Porfornlng  Adnlalstrstlve  Outlet 

Sou  effective  Is  each  soldier  In  preparing  foras  conpletely.  neatly,  and 
accurately;  obtaining  needed  ferns  before  departing  on  ceunttneet;  turning  In 
foras  to  proper  persons? 


1.  Self-tecovorlng  Vehicles 

Nou  effective  Is  each  soldier  In  taking  correct  action  uhca  vehicles  are 
disabled;  using  ulnch  or  other  equipment  to  perform  vehicle  self-recovery; 
fellevlng  proper  procedures  uben  recovering  or  toeing  vehicles? 


J.  Safety-* laded nets 

leu  effective  Is  each  soldier  la  knovlag  and  following  safety  procedures;  being 
alert  to  possible  dangerous  situations  and  taking  steps  to  avoid  thee;  using 
proper  safety  equipment? 


K.  Performing  Dispatcher  Duties 

Nov  effective  Is  each  soldier  In  dispatching  other  NTOs? 


91A  Medical  Specialist  Behavioral-gerf prance  Categories 

The  content  of  four  first  tour  performance  categories  was  unchanged  for 
second  tour.  For  the  other  four  categories,  modifications  were  Hade  to 
reflect  additional  second  tour  perfomance  requlrements/expec  tat  Ions  (e.g., 
ensuring  that  routine  PMCS  Is  performed,  ensuring  that  adequate  supplier 
were  maintained,  providing  patient  care  without  supervision,  taking  a 
leadership  role  In  emergency  situations).  No  MOS-speciflc  supervision 
categories  were  added  for  the  9IA/B  HOS. 

The  first  and  second  tour  perfomance  category  names  are  shown  below; 
the  second  tour  perfomance  category  definitions  are  listed  on  the  next 
page. 


FIRST  TOUR  PERFORMANCE  CATEGORIES 

Maintain  and  Operate  Amy  Medical 
Vehicles  and  Equipment 

Maintain  Accountability  of 
Medical  Supply/Equipment 

Keeping  Medical  Records 

Arranging  for  Transportation/ 
Transport  Injured  Personnel 

Dispensing  Medications 

Preparing/Maintaining  Field 
Site/Clinic  Facilities 

Respond  Medical  Emergencies 

Provide  Routine  and  Ongoing 
Patient  Care 

Responding  to  Emergencies 

Provide  Health  Care  and  Health 
Maintenance  Instruction 


SECGMD  TOUR  PERFORMANCE  CATEGORIES 

Maintain  and  Operate  Amy  Medical 
Vehicles  and  Equipment 

Maintain  Accountability  of 
Medical  Supply/Equipment 

Keeping  Medical  Records 

Arranging  for  Transportation/ 
Transport  Injured  Personnel 

Dispensing  Medications 

Preparing/Maintaining  Field 
Site/Clinic  Facilities 

Respond  Medical  Emergencies 

Provide  Routine  and  Ongoing 

Responding  to  Emergencies 

Provide  Health  Care  and  Health 
Maintenance  Instruction 
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SECOND  TOUR  DOS  PERFORMANCE  CATEGORY  MANES  AMD  DEFINITIONS  FOR  NOS 

MEDICAL  SPECIALIST  (I1A) 


A.  Maintaining  and  Operatic?  Amy  Radical  Vmhttlm*  and  Equipment 

Now  effective  Is  each  soldier  la  Inspecting  and  Maintaining  Amy  medical 
vehicles  and  equipment  tc  aasura  that  tNay  ara  la  misatoe-medy  ttataa.  aad 
operating  vehicles  la  a  uf«  and  efficient  manner? 


I.  Maintaining  Accountability  of  Radical  Supplies  and  Equipment 

Row  affactlv*  It  each  toldlar  1a  keeping  aslttlag  wed  lea  I  suppl les/equlpmaat 
wel 1 -stocked  aad  properly  stored  and  secured? 


C.  Keeping  Radical  Records 

Now  affective  It  each  soldier  la  completing  aad  aataulaiag  patients'  records 
(e.g..  wed leal  history.  current  medication.  treatment,  etc.)  and  ensuring  that 
all  fomt  ara  accurate  and  up-to-date? 


0.  Arranging  for  Transportation  and/or  Transporting  Injured  Personae! 

Now  affective  Is  each  soldier  in  transporting  patients  safety  end  properly  aad 
arranging  for  transportation  of  Injured  personnel? 


C.  Dispensing  Medications 

Now  effective  Is  each  soldier  In  detenstnlng  or  Identifying  the  correct 
medication  end  administering  medications  at  Instructed  or  as  required! 


Preparing  and  Maintaining  Field  Site  or  Clinic  Facilities  ta  the  Fiald 


—  «. •SCi'vg  !■  SSCm  SCl  dlsr  in  preparing  for  tf*e  field,  ••♦♦lag  •*>  sad 
intalnlng  field  facilities,  end  ensuring  sanitary  conditions? 


RIM  V 


C.  Providing  Routine  and  Ongoing  Patient  Cere 

Now  effective  Is  each  soldier  In  providing  csre  for  non-eaerooncy  Injuries 
or  Illnesses,  aonltortng  patients,  and  assisting  the  physical  as  required? 

N.  Responding  to  Emergency  Situation 

Now  effective  is  each  soldier  in  responding  quickly  end  Immediately  to  life* 
threatening  situations  at  accident  sites.  In  the  field,  or  In  emergency  rooms? 


I.  Providing  Health  Cere  end  Health  Maintenance  Instruction  ta  Army  Personnel 

How  effective  Is  each  soldier  in  training  other  medics  on  proper  injury  or  Illness 
prevention  techniques,  end  training  soldiers  on  health  maintenance  procedures? 
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956  Military  Police. Behavioral  Performance  Categories 

The  content  of  Mvon  (out  of  11)  first  tour  performance  categories  was 
unchanged  for  second  tour.  Three  of  the  first  tour  categories  were  modified 
to  reflect  additional  second  tour  performance  reouirements/expectatlons 
(e,g..  planning  the  safe  passage  of  equipment  and  personnel,  only  using  the 
necessary  degree  of  force,  becoming  familiar  with  more  of  the  patrol  area). 
One  first  tour  performance  category,  'Courage  and  Proficiency  in  Battle*  was 
dropped,  because  the  content  of  this  category  seemed  to  be  adequately 
covered  by  other  second  tour  95B  dimensions. 

Four  new  categories  were  added  to  the  second  tour  scales.  Three  of 
these,  "Fighting  Positions,*  'Battlefield  Circulation  Control,*  and  'Enemy 
Prisoners  of  Her,*  were  added  to  reflect  changes  In  HQS  doctrine  since  the 
time  the  first  tour  scales  were  developed.  These  categories  would  also  be 
applicable  to  first  tour  performance.  The  fourth  category  addition  was  an 
MOS-specif 1c  supervision  category,  which  contains  behaviors  such  as  briefing 
soldiers  concerning  the  mission,  leading  teams  in  the  field,  and  using 
appropriate  strategies  and  tactics. 


FIRST  TOUR  PERFORMANCE  CATEGORIES 
Traffic  Control /Enforcement 
Providing  Security 
Investigate  Crimes/Make  Arrest 
Patrolling 

Promote  Public  Image  of  MPs 
Interpersonal  Communication 
Respond  Medical  Emergencies 
Navigation 

Avoiding  Enemy  Detection 
Use  of  Weapons/Equipment 
Courage/Prof 1  in  Battle 


SECOND  TOUR  PERFORMANCE  CATEGORIES 
Traffic  Control /Enforcement 
Providing  Security 
Investigate  Crimes/Make  Arrest 
Patrolling 

Promote  Public  Image  of  MPs 
Interpersonal  Communication 
Respond  Medical  Emergencies 
Navigation 

Avoiding  Enemy  Detection 
Use  of  Veapons/Equipsent 

Fighting  Position 
Battlefield  Clr  Control 
Enemy  Prisoners  of  War 
Lead  Team  ir  Tact  Environment 
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SECOND  TOUR  NOS  PERFORMANCE  CATEGORY  NAMES  AMD  DEFINITIONS  FOR  NOS: 

MILITARY  POLICE  <»») 


A.  Traffic  Control  and  enforcement 

Now  ef fectivo  is  tacji  soldier  In  extent  teg  traffic  control  and  enforcing 
traffic  laws  and  parting  roles? 

I.  Providing  Security 

■on  effective  it  nc»  to  Id  tar  in  pleneing  far  and  providing  physical  security 
and  at cor t  security? 

C.  Iavastlgatiag  Criaat  and  Hating  Apprahewlen* 

Bo*  affactira  it  each  toldlar  in  gathering  information  on  criminal  activity, 
aaking  apprehensions,  and  raportiAg  on  criaatt 

0.  Patrolling 

low  affactiva  it  aach  toldiar  In  datacting.  raipoadlng  to.  and  investigating 
tatpactod  crlolnal  activities  vhlle  on  patrol? 

C.  Loading  the  Teas  In  a  Tactical  Envlrooaant 

Bob  affactiva  it  aacti  toldiar  in  loading  the  taaa  and  briefing  subordinates 
on  tba  aittionT 

F.  Promoting  tbo  Public  Inaga  of  tbe  Nllltary  Police 

low  affactiva  it  aach  toldiar  In  displaying  professional  I sb.  providing 
assistance,  and  otherwise  promoting  tba  public  inaga  of  tba  nilftary  police 
while  on- the- Job  and  off«duty? 

'  S.  Interpersonal  Coaaoni cat  ions  Skills 

Bow  affective  is  each  soldier  in  renaining  cals  and  dealing  affectively  with 
disturbances  and  with  individuals  who  are  uosat.  angry,  or  potentially  violent? 

N.  Responding  to  Radical  Eaergenclet 

Bow  effective  It  each  soldier  In  responding  to  nodical  emergencies  and  accidents? 

I.  navigation 

Now  affactiva  it  aach  sotdiar  in  using  navigational  equipment  and  navigating 
ia  the  field? 

J.  Avoiding  Enemy  Detection 

Bow  affective  is  each  toldiar  In  avoiding  tneny  dataction  during  movement  and  In 
astablished  defensive  positions  vhilt  ia  tba  fiatd? 

K.  Bsa  or  Meapont  and  other  Equipment 

Bow  effective  it  each  soldier  ia  vslng  weapons  and  other  equipment  safely  and 
proficiently? 

L.  Fighting  Positions 

Now  affactiva  is  each  seidler  in  trepsrlng  a  fighting  petition,  range  cards,  and 
sector  sketches? 

H.  battlefield  Circulation  Control 

Now  effective  Is  each  toldiar  la  performing  battlefield  circulation  control  (ICC) 
in  a  tactical  environment? 

I.  Enemy  Prisoners  of  Her 

Now  affactiva  Is  aach  soldier  In  processing  eneny  prisoners  of  war  during  field 
exercises  or  In  combat? 
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